Optimizing Human-Computer Interaction: By integrating facial emotion detection into computer systems, we aim to create more responsive and intuitive interactions, making technology more attuned to human needs and emotional states.
Enhancing Safety and Security: Through the detection of emotional cues, such as stress or fatigue, particularly in critical sectors like automotive and security systems, we aim to proactively address safety concerns and potentially prevent accidents or security breaches.
Improving Healthcare and Mental Health Monitoring: By identifying emotional changes over time, we can offer better insights into mental health and patient care. This could lead to early detection of conditions like depression or anxiety, and improve the overall quality of care in healthcare settings.
Advancing Retail, Entertainment, and Marketing: Understanding customer emotions in real-time can revolutionize these sectors by providing enhanced user experiences, personalized content, and improved customer service.
Developing Emotional Intelligence in Robotics and AI: The goal is to endow robots and AI systems with the ability to recognize and respond to human emotions, paving the way for more empathetic and sophisticated interactions between humans and machines.
The data set consists of 3 folders, i.e., 'test', 'train', and 'validation'. Each of these folders has four subfolders:
‘happy’: Images of people who have happy facial expressions.
‘sad’: Images of people with sad or upset facial expressions.
‘surprise’: Images of people who have shocked or surprised facial expressions.
‘neutral’: Images of people showing no prominent emotion in their facial expression at all.
NOTE: Please use Google Colab from your browser for this notebook. Google.colab is NOT a library that can be downloaded locally on your device.
from google.colab import drive
drive.mount('/content/drive')
Mounted at /content/drive
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import MinMaxScaler
import tensorflow as tf
from tensorflow.keras.preprocessing.image import load_img, img_to_array
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.layers import Dense, Input, Dropout, GlobalAveragePooling2D, Flatten, Conv2D, BatchNormalization, Activation, MaxPooling2D, LeakyReLU
from tensorflow.keras.models import Model, Sequential
import zipfile
#Checking Tensorflow version
print(tf.__version__)
2.15.0
Note:
# Loading dataset to variable 'path'
path = '/content/drive/MyDrive/Useful Docs/Facial_emotion_images.zip'
# Extracting dataset from zipfile
with zipfile.ZipFile(path, 'r') as zip_ref:
zip_ref.extractall()
# Loading dataset to variable 'path' if working locally
path = '/Users/ceb/Downloads/TEMP/Facial_emotion_images'
import os
folder_path = "Facial_emotion_images/"
emotions_data = {}
for root, dirs, files in os.walk(folder_path):
for dir_name in dirs:
# Split the root to get the main folder and subfolder names
parts = root.split(os.sep)
if len(parts) > 1:
main_folder = parts[-1]
sub_folder = dir_name
# Initialize nested dictionary if not already present
if main_folder not in emotions_data:
emotions_data[main_folder] = {}
# Initialize the sub-folder dictionary
dir_path = os.path.join(root, dir_name)
emotions_data[main_folder][sub_folder] = {'path': dir_path, 'count': 0}
from PIL import Image
import os
folder_path = "Facial_emotion_images/"
emotions_data = {}
# Initialize Nested Emotion Data Dictionary
for root, dirs, files in os.walk(folder_path):
for dir_name in dirs:
parts = root.split(os.sep)
if len(parts) > 1:
main_folder = parts[-1]
sub_folder = dir_name
if main_folder not in emotions_data:
emotions_data[main_folder] = {}
dir_path = os.path.join(root, dir_name)
emotions_data[main_folder][sub_folder] = {'path': dir_path, 'count': 0, 'size': None}
# Count Files and Determine Size
for main_folder in emotions_data:
for sub_folder in emotions_data[main_folder]:
dir_path = emotions_data[main_folder][sub_folder]['path']
file_count = 0
if os.path.isdir(dir_path):
for file in os.listdir(dir_path):
if file.lower().endswith(('.png', '.jpg', '.jpeg')):
if file_count == 0: # Check size for the first image
with Image.open(os.path.join(dir_path, file)) as img:
emotions_data[main_folder][sub_folder]['size'] = img.size
file_count += 1
emotions_data[main_folder][sub_folder]['count'] = file_count
# Print the updated information
print(f"Folder: {main_folder}, Emotion: {sub_folder}, Count: {emotions_data[main_folder][sub_folder]['count']}, Size: {emotions_data[main_folder][sub_folder]['size']}, Path: {dir_path}")
Folder: , Emotion: test, Count: 0, Size: None, Path: Facial_emotion_images/test Folder: , Emotion: train, Count: 0, Size: None, Path: Facial_emotion_images/train Folder: , Emotion: validation, Count: 0, Size: None, Path: Facial_emotion_images/validation Folder: test, Emotion: happy, Count: 32, Size: (48, 48), Path: Facial_emotion_images/test/happy Folder: test, Emotion: sad, Count: 32, Size: (48, 48), Path: Facial_emotion_images/test/sad Folder: test, Emotion: surprise, Count: 32, Size: (48, 48), Path: Facial_emotion_images/test/surprise Folder: test, Emotion: neutral, Count: 32, Size: (48, 48), Path: Facial_emotion_images/test/neutral Folder: train, Emotion: happy, Count: 3976, Size: (48, 48), Path: Facial_emotion_images/train/happy Folder: train, Emotion: sad, Count: 3982, Size: (48, 48), Path: Facial_emotion_images/train/sad Folder: train, Emotion: surprise, Count: 3173, Size: (48, 48), Path: Facial_emotion_images/train/surprise Folder: train, Emotion: neutral, Count: 3978, Size: (48, 48), Path: Facial_emotion_images/train/neutral Folder: validation, Emotion: happy, Count: 1825, Size: (48, 48), Path: Facial_emotion_images/validation/happy Folder: validation, Emotion: sad, Count: 1139, Size: (48, 48), Path: Facial_emotion_images/validation/sad Folder: validation, Emotion: surprise, Count: 797, Size: (48, 48), Path: Facial_emotion_images/validation/surprise Folder: validation, Emotion: neutral, Count: 1216, Size: (48, 48), Path: Facial_emotion_images/validation/neutral
# Confirming whether the structure works, it can be reused later in the code
print(emotions_data['train']['sad']['path'])
print(emotions_data['train']['sad']['size'])
print(emotions_data['train']['sad']['count'])
Facial_emotion_images/train/sad (48, 48) 3982
# Store image size to be reused since all images are 48x48 pixels
img_size = emotions_data['train']['sad']['size']
# Reusing variables from earlier data exploration
emotion = 'happy'
path = emotions_data['train'][emotion]['path']
# Plotting images for visualisation
plt.figure(figsize= (8,8))
for i in range(1, 10, 1):
plt.subplot(3, 3, i)
img = load_img(path + "/" +
os.listdir(path)[i], target_size = (img_size))
plt.imshow(img)
plt.show()
Observations and Insights:The 'happy' images predominantly feature smiles, with the corners of the mouth turned upwards and eyes often crinkling. A key distinguishing feature is the genuine warmth in the eyes, setting it apart from neutral or sad expressions. However, some challenging images include subtle smiles, which could be confused with a neutral expression, especially in lower-resolution or unclear images. Some images are also partially ocluded which could be a challenge for the Neural Network.
# Reusing variables from earlier data exploration
emotion = 'sad'
path = emotions_data['train'][emotion]['path']
# Plotting images for visualisation
plt.figure(figsize= (8,8))
for i in range(1, 10, 1):
plt.subplot(3, 3, i)
img = load_img(path + "/" +
os.listdir(path)[i], target_size = (img_size))
plt.imshow(img)
plt.show()
Observations and Insights: Sad expressions are characterized by downturned mouths, furrowed brows, and sometimes teary eyes. A unique aspect is the overall downward trajectory of facial features. Challenges arise in distinguishing subtle sad expressions from neutral ones, as some individuals may exhibit less pronounced sadness cues.
# Reusing variables from earlier data exploration
emotion = 'neutral'
path = emotions_data['train'][emotion]['path']
# Plotting images for visualisation
plt.figure(figsize= (8,8))
for i in range(1, 10, 1):
plt.subplot(3, 3, i)
img = load_img(path + "/" +
os.listdir(path)[i], target_size = (img_size))
plt.imshow(img)
plt.show()
Observations and Insights: Neutral expressions are marked by a lack of significant emotional indicators. The mouth is typically relaxed and straight, and the eyes are open without notable crinkling or squinting. The challenge in this category is differentiating between subtle emotional expressions and truly neutral ones, as minimal cues can sometimes be misinterpreted.
# Reusing variables from earlier data exploration
emotion = 'surprise'
path = emotions_data['train'][emotion]['path']
# Plotting images for visualisation
plt.figure(figsize= (8,8))
for i in range(1, 10, 1):
plt.subplot(3, 3, i)
img = load_img(path + "/" +
os.listdir(path)[i], target_size = (img_size))
plt.imshow(img)
plt.show()
Observations and Insights: Surprised expressions are easily identifiable by wide-open eyes and often an open mouth. The uniqueness lies in the sense of suddenness and exaggeration in the features. The difficulty is in differentiating between positive surprise and other high-arousal emotions, like happy, in some images where context and lesser cues are present.
# 'category' will be 'train', 'test' and 'validation'
for category in emotions_data:
for emotion in emotions_data[category]:
if emotions_data[category][emotion]['count'] != 0:
print("The {} dataset has {} {} images.".format(category, emotions_data[category][emotion]['count'], emotion))
The test dataset has 32 happy images. The test dataset has 32 sad images. The test dataset has 32 surprise images. The test dataset has 32 neutral images. The train dataset has 3976 happy images. The train dataset has 3982 sad images. The train dataset has 3173 surprise images. The train dataset has 3978 neutral images. The validation dataset has 1825 happy images. The validation dataset has 1139 sad images. The validation dataset has 797 surprise images. The validation dataset has 1216 neutral images.
# Calculating the ratio between largest dataset and smallest. Other datasets sizes are similar to 'sad' dataset
surprise_ratio = emotions_data['train']['surprise']['count']/emotions_data['train']['sad']['count']
print("The 'surprise' dataset is {:.1f}% smaller than 'sad' dataset".format((1-surprise_ratio)*100))
The 'surprise' dataset is 20.3% smaller than 'sad' dataset
# Visualising the validation and test datasets
emotion = 'happy'
path = emotions_data['validation'][emotion]['path']
# Plotting images for visualisation
plt.figure(figsize= (8,8))
for i in range(1, 10, 1):
plt.subplot(3, 3, i)
img = load_img(path + "/" +
os.listdir(path)[i], target_size = (img_size))
plt.imshow(img)
plt.show()
# Visualising the validation and test datasets
emotion = 'sad'
path = emotions_data['validation'][emotion]['path']
# Plotting images for visualisation
plt.figure(figsize= (8,8))
for i in range(1, 10, 1):
plt.subplot(3, 3, i)
img = load_img(path + "/" +
os.listdir(path)[i], target_size = (img_size))
plt.imshow(img)
plt.show()
# Visualising the validation and test datasets
emotion = 'neutral'
path = emotions_data['validation'][emotion]['path']
# Plotting images for visualisation
plt.figure(figsize= (8,8))
for i in range(1, 10, 1):
plt.subplot(3, 3, i)
img = load_img(path + "/" +
os.listdir(path)[i], target_size = (img_size))
plt.imshow(img)
plt.show()
# Visualising the validation and test datasets
emotion = 'surprise'
path = emotions_data['validation'][emotion]['path']
# Plotting images for visualisation
plt.figure(figsize= (8,8))
for i in range(1, 10, 1):
plt.subplot(3, 3, i)
img = load_img(path + "/" +
os.listdir(path)[i], target_size = (img_size))
plt.imshow(img)
plt.show()
# Visualising the validation and test datasets
emotion = 'happy'
path = emotions_data['test'][emotion]['path']
# Plotting images for visualisation
plt.figure(figsize= (8,8))
for i in range(1, 10, 1):
plt.subplot(3, 3, i)
img = load_img(path + "/" +
os.listdir(path)[i], target_size = (img_size))
plt.imshow(img)
plt.show()
# Visualising the validation and test datasets
emotion = 'sad'
path = emotions_data['test'][emotion]['path']
# Plotting images for visualisation
plt.figure(figsize= (8,8))
for i in range(1, 10, 1):
plt.subplot(3, 3, i)
img = load_img(path + "/" +
os.listdir(path)[i], target_size = (img_size))
plt.imshow(img)
plt.show()
# Visualising the validation and test datasets
emotion = 'neutral'
path = emotions_data['test'][emotion]['path']
# Plotting images for visualisation
plt.figure(figsize= (8,8))
for i in range(1, 10, 1):
plt.subplot(3, 3, i)
img = load_img(path + "/" +
os.listdir(path)[i], target_size = (img_size))
plt.imshow(img)
plt.show()
# Visualising the validation and test datasets
emotion = 'surprise'
path = emotions_data['test'][emotion]['path']
# Plotting images for visualisation
plt.figure(figsize= (8,8))
for i in range(1, 10, 1):
plt.subplot(3, 3, i)
img = load_img(path + "/" +
os.listdir(path)[i], target_size = (img_size))
plt.imshow(img)
plt.show()
# Checking image sizes
print("Happy images have the size of", emotions_data['train']['happy']['size'])
print("Happy images have the size of", emotions_data['train']['sad']['size'])
print("Happy images have the size of", emotions_data['train']['neutral']['size'])
print("Happy images have the size of", emotions_data['train']['surprise']['size'])
Happy images have the size of (48, 48) Happy images have the size of (48, 48) Happy images have the size of (48, 48) Happy images have the size of (48, 48)
Further Data Exploration has revealed:
Observations and Insights: The noted imbalance in the 'surprise' dataset, being significantly smaller than the 'sad' dataset (20.3% smaller), indicates a skew in the data. The test dataset containing 32 happy images also suggests potential imbalances. While some degree of class imbalance is common in real-world datasets, the extent of this discrepancy might pose challenges.
Class Distribution: The classes are not equally distributed. The 20.3% smaller 'surprise' dataset compared to 'sad' suggests a notable imbalance. Such disparities can impact the model's performance, potentially leading it to be less accurate in identifying less-represented emotions.
Impact of Imbalance: This imbalance could be problematic as it may cause the neural network to be biased towards recognizing emotions that are more frequently represented in the training data. It may underperform in identifying 'surprise' due to the lack of representation in the data.
In this section, we are creating data loaders that we will use as inputs to our Neural Network.
# Batch size to be used for training
batch_size = 32
# Create an instance of ImageDataGenerator
datagen = ImageDataGenerator(horizontal_flip = True,
brightness_range=(0.,2.),
rescale=1./255,
shear_range=0.3)
# Creating dataloader for train for RGB dataset
train_set_rgb = datagen.flow_from_directory(folder_path + "train",
target_size = (img_size),
color_mode = 'rgb',
batch_size = batch_size,
class_mode = 'categorical',
shuffle = True)
# Creating dataloader for train for Grayscale dataset
train_set_gray = datagen.flow_from_directory(folder_path + "train",
target_size = (img_size),
color_mode = 'grayscale',
batch_size = batch_size,
class_mode = 'categorical',
shuffle = True)
# Creating dataloader for validation for RGB dataset
validation_set_rgb = datagen.flow_from_directory(folder_path + "validation",
target_size = (img_size),
color_mode = 'rgb',
batch_size = batch_size,
class_mode = 'categorical',
shuffle = True)
# Creating dataloader for validation for Grayscale dataset
validation_set_gray = datagen.flow_from_directory(folder_path + "validation",
target_size = (img_size),
color_mode = 'grayscale',
batch_size = batch_size,
class_mode = 'categorical',
shuffle = True)
# Creating dataloader for test for RGB dataset
test_set_rgb = datagen.flow_from_directory(folder_path + "test",
target_size = (img_size),
color_mode = 'rgb',
batch_size = batch_size,
class_mode = 'categorical',
shuffle = True)
# Creating dataloader for test for Grayscale dataset
test_set_gray = datagen.flow_from_directory(folder_path + "test",
target_size = (img_size),
color_mode = 'grayscale',
batch_size = batch_size,
class_mode = 'categorical',
shuffle = True)
Found 15109 images belonging to 4 classes. Found 15109 images belonging to 4 classes. Found 4977 images belonging to 4 classes. Found 4977 images belonging to 4 classes. Found 128 images belonging to 4 classes. Found 128 images belonging to 4 classes.
# Fixing the seed for random number generators
fixed_seed = 42
#Creating a function as we'll be resetting the seed multiple times throughout the notebook
def reset_seed(seed):
np.random.seed(seed)
import random
random.seed(seed)
tf.random.set_seed(seed)
print("Random number generators seed has been set to", seed)
import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout, Activation, BatchNormalization, Flatten, Conv2D, MaxPooling2D, LeakyReLU
from tensorflow.keras.losses import categorical_crossentropy
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.optimizers import legacy
from tensorflow.keras.utils import to_categorical
from tensorflow.keras import backend
# Clearing backend
backend.clear_session()
# Reset seed for number generators using reset_seed() function
reset_seed(fixed_seed)
Random number generators seed has been set to 42
# Defining Base NN
def cnn_model_1(colour_layers):
cnn_model = Sequential()
# First Convolutional layer with 64 filters and the kernel size of 3x3, 'same' padding and input shape = (48, 48, 3 - RGB colours)
cnn_model.add(Conv2D(filters = 64, kernel_size = (2, 2), padding = "same", input_shape = (img_size[0], img_size[1], colour_layers), activation = 'relu'))
# A max-pooling layer with a pool size of 2x2
cnn_model.add(MaxPooling2D(pool_size = (2, 2)))
# Dropout layer with the rate equal to 0.2
cnn_model.add(Dropout(0.2))
# Second Convolutional layer with 32 filters and the kernel size of 2x2 with 'same' padding
cnn_model.add(Conv2D(filters = 32, kernel_size = (2, 2), padding = "same", activation = 'relu'))
# A max-pooling layer with a pool size of 2x2
cnn_model.add(MaxPooling2D(pool_size = (2, 2)))
# Dropout layer with the rate equal to 0.2
cnn_model.add(Dropout(0.2))
# Third Convolutional layer with 32 filters and the kernel size of 2x2 with 'same' padding
cnn_model.add(Conv2D(filters = 32, kernel_size = (2, 2), padding = "same", activation = 'relu'))
# A max-pooling layer with a pool size of 2x2
cnn_model.add(MaxPooling2D(pool_size = (2, 2)))
# Dropout layer with the rate equal to 0.2
cnn_model.add(Dropout(0.2))
# Flatten the output from the previous layer
cnn_model.add(Flatten())
# Dense layer with 512 nodes
cnn_model.add(Dense(512, activation = 'relu'))
# Dropout layer with the rate equal to 0.4
cnn_model.add(Dropout(0.4))
# Final output layer with nodes equal to the number of classes (sad, happy, neutral and surprised) and 'softmax' as the activation function
cnn_model.add(Dense(4, activation = 'softmax'))
# Compiling the CNN model with categorical crossentropy as loss function, Adam Optimizer
# with 0.001 learning rate, and set metrics set to 'accuracy'.
cnn_model.compile(loss='categorical_crossentropy',
optimizer=legacy.Adam(learning_rate = 0.001),
metrics=['accuracy'])
return cnn_model
from keras.callbacks import ModelCheckpoint, EarlyStopping, ReduceLROnPlateau
# ModelCheckpoint is used to save the model at certain intervals
checkpoint = ModelCheckpoint("model1.h5", # file path where the model will be saved
monitor='val_accuracy', # means the callback monitors the validation accuracy
verbose=1, # model is saved only when the metric (val_acc) has improved
save_best_only=True,
mode='max')
# Stop the training process early if a monitored metric has stopped improving
early_stopping = EarlyStopping(monitor = 'val_loss', # monitors the validation loss
min_delta = 0, # any positive change is considered as an improvement
patience = 3, # training will be stopped if no improvement is seen for 3 epochs
verbose = 1,
restore_best_weights = True # weights are reverted to best value after early stopping
)
# Reduces the learning rate when a metric has stopped improving
reduce_learningrate = ReduceLROnPlateau(monitor = 'val_loss', # monitor the validation loss
factor = 0.2, # learning rate reduced to 20% when reduction is triggered
patience = 3, # reduction happens after 3 epochs with no improvement
verbose = 1,
min_delta = 0.0001)
# Configured callbacks will be passed to the model during training
callbacks_list = [early_stopping, checkpoint, reduce_learningrate]
# Number of iterations over the entire dataset that the training process should run
epochs = 20
# Building the model
model1 = cnn_model_1(3)
# Print first model's summary
model1.summary()
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 48, 48, 64) 832
max_pooling2d (MaxPooling2 (None, 24, 24, 64) 0
D)
dropout (Dropout) (None, 24, 24, 64) 0
conv2d_1 (Conv2D) (None, 24, 24, 32) 8224
max_pooling2d_1 (MaxPoolin (None, 12, 12, 32) 0
g2D)
dropout_1 (Dropout) (None, 12, 12, 32) 0
conv2d_2 (Conv2D) (None, 12, 12, 32) 4128
max_pooling2d_2 (MaxPoolin (None, 6, 6, 32) 0
g2D)
dropout_2 (Dropout) (None, 6, 6, 32) 0
flatten (Flatten) (None, 1152) 0
dense (Dense) (None, 512) 590336
dropout_3 (Dropout) (None, 512) 0
dense_1 (Dense) (None, 4) 2052
=================================================================
Total params: 605572 (2.31 MB)
Trainable params: 605572 (2.31 MB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________
# Visualize the model as blocks
from tensorflow.keras.utils import plot_model
plot_model(model1, to_file='model.png', show_shapes=True, show_layer_names=True)
# Fitting the model with required parameters above, saving model into history variable
history = model1.fit(
train_set_rgb,
steps_per_epoch = train_set_rgb.samples // train_set_rgb.batch_size,
validation_data = validation_set_rgb,
validation_steps = validation_set_rgb.samples // validation_set_rgb.batch_size,
epochs = 20,
verbose = 1,
callbacks = callbacks_list
)
Epoch 1/20 472/472 [==============================] - ETA: 0s - loss: 1.3347 - accuracy: 0.3343 Epoch 1: val_accuracy improved from -inf to 0.46310, saving model to model1.h5 472/472 [==============================] - 15s 32ms/step - loss: 1.3347 - accuracy: 0.3343 - val_loss: 1.2183 - val_accuracy: 0.4631 - lr: 0.0010 Epoch 2/20 5/472 [..............................] - ETA: 11s - loss: 1.1861 - accuracy: 0.5312
/Users/ceb/anaconda3/lib/python3.11/site-packages/keras/src/engine/training.py:3103: UserWarning: You are saving your model as an HDF5 file via `model.save()`. This file format is considered legacy. We recommend using instead the native Keras format, e.g. `model.save('my_model.keras')`.
saving_api.save_model(
472/472 [==============================] - ETA: 0s - loss: 1.1885 - accuracy: 0.4742 Epoch 2: val_accuracy improved from 0.46310 to 0.49859, saving model to model1.h5 472/472 [==============================] - 15s 31ms/step - loss: 1.1885 - accuracy: 0.4742 - val_loss: 1.1562 - val_accuracy: 0.4986 - lr: 0.0010 Epoch 3/20 472/472 [==============================] - ETA: 0s - loss: 1.1079 - accuracy: 0.5114 Epoch 3: val_accuracy improved from 0.49859 to 0.57802, saving model to model1.h5 472/472 [==============================] - 15s 32ms/step - loss: 1.1079 - accuracy: 0.5114 - val_loss: 1.0088 - val_accuracy: 0.5780 - lr: 0.0010 Epoch 4/20 471/472 [============================>.] - ETA: 0s - loss: 1.0537 - accuracy: 0.5370 Epoch 4: val_accuracy did not improve from 0.57802 472/472 [==============================] - 16s 33ms/step - loss: 1.0536 - accuracy: 0.5370 - val_loss: 0.9875 - val_accuracy: 0.5712 - lr: 0.0010 Epoch 5/20 471/472 [============================>.] - ETA: 0s - loss: 1.0129 - accuracy: 0.5593 Epoch 5: val_accuracy improved from 0.57802 to 0.59637, saving model to model1.h5 472/472 [==============================] - 15s 32ms/step - loss: 1.0128 - accuracy: 0.5591 - val_loss: 0.9513 - val_accuracy: 0.5964 - lr: 0.0010 Epoch 6/20 470/472 [============================>.] - ETA: 0s - loss: 0.9952 - accuracy: 0.5742 Epoch 6: val_accuracy improved from 0.59637 to 0.59859, saving model to model1.h5 472/472 [==============================] - 15s 32ms/step - loss: 0.9952 - accuracy: 0.5744 - val_loss: 0.9347 - val_accuracy: 0.5986 - lr: 0.0010 Epoch 7/20 471/472 [============================>.] - ETA: 0s - loss: 0.9745 - accuracy: 0.5829 Epoch 7: val_accuracy improved from 0.59859 to 0.61613, saving model to model1.h5 472/472 [==============================] - 15s 32ms/step - loss: 0.9744 - accuracy: 0.5826 - val_loss: 0.9284 - val_accuracy: 0.6161 - lr: 0.0010 Epoch 8/20 471/472 [============================>.] - ETA: 0s - loss: 0.9541 - accuracy: 0.5880 Epoch 8: val_accuracy did not improve from 0.61613 472/472 [==============================] - 15s 32ms/step - loss: 0.9537 - accuracy: 0.5882 - val_loss: 0.9179 - val_accuracy: 0.6093 - lr: 0.0010 Epoch 9/20 472/472 [==============================] - ETA: 0s - loss: 0.9440 - accuracy: 0.5959 Epoch 9: val_accuracy did not improve from 0.61613 472/472 [==============================] - 15s 31ms/step - loss: 0.9440 - accuracy: 0.5959 - val_loss: 0.9320 - val_accuracy: 0.6153 - lr: 0.0010 Epoch 10/20 471/472 [============================>.] - ETA: 0s - loss: 0.9281 - accuracy: 0.6023 Epoch 10: val_accuracy improved from 0.61613 to 0.63690, saving model to model1.h5 472/472 [==============================] - 15s 32ms/step - loss: 0.9274 - accuracy: 0.6026 - val_loss: 0.8957 - val_accuracy: 0.6369 - lr: 0.0010 Epoch 11/20 470/472 [============================>.] - ETA: 0s - loss: 0.9142 - accuracy: 0.6110 Epoch 11: val_accuracy improved from 0.63690 to 0.64214, saving model to model1.h5 472/472 [==============================] - 15s 32ms/step - loss: 0.9138 - accuracy: 0.6109 - val_loss: 0.8720 - val_accuracy: 0.6421 - lr: 0.0010 Epoch 12/20 471/472 [============================>.] - ETA: 0s - loss: 0.9040 - accuracy: 0.6149 Epoch 12: val_accuracy improved from 0.64214 to 0.64778, saving model to model1.h5 472/472 [==============================] - 15s 32ms/step - loss: 0.9044 - accuracy: 0.6144 - val_loss: 0.8334 - val_accuracy: 0.6478 - lr: 0.0010 Epoch 13/20 472/472 [==============================] - ETA: 0s - loss: 0.8861 - accuracy: 0.6217 Epoch 13: val_accuracy improved from 0.64778 to 0.65202, saving model to model1.h5 472/472 [==============================] - 15s 32ms/step - loss: 0.8861 - accuracy: 0.6217 - val_loss: 0.8418 - val_accuracy: 0.6520 - lr: 0.0010 Epoch 14/20 471/472 [============================>.] - ETA: 0s - loss: 0.8841 - accuracy: 0.6239 Epoch 14: val_accuracy did not improve from 0.65202 472/472 [==============================] - 15s 33ms/step - loss: 0.8843 - accuracy: 0.6239 - val_loss: 0.8714 - val_accuracy: 0.6433 - lr: 0.0010 Epoch 15/20 472/472 [==============================] - ETA: 0s - loss: 0.8712 - accuracy: 0.6328Restoring model weights from the end of the best epoch: 12. Epoch 15: val_accuracy did not improve from 0.65202 Epoch 15: ReduceLROnPlateau reducing learning rate to 0.00020000000949949026. 472/472 [==============================] - 15s 32ms/step - loss: 0.8712 - accuracy: 0.6328 - val_loss: 0.8391 - val_accuracy: 0.6510 - lr: 0.0010 Epoch 15: early stopping
from keras.models import load_model
# Load the saved model if notebook session gets disconnected
model1 = load_model("model1.h5")
# Evaluate the model on the test data using
test_loss, test_accuracy = model1.evaluate(test_set_rgb, verbose=1)
# Print the results
print("Test Loss:", test_loss)
print("Test Accuracy:", test_accuracy)
4/4 [==============================] - 0s 24ms/step - loss: 0.8422 - accuracy: 0.6094 Test Loss: 0.8422161936759949 Test Accuracy: 0.609375
# Importing relevant libraries
import seaborn as sns
import matplotlib.pyplot as plt
# Plotting the Training and Validation accuracies of CNN model 2
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.title('CNN Model 1 Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc = 'upper left')
# Display the plot
plt.show()
# Clearing backend
backend.clear_session()
# Reset seed for number generators using reset_seed() function
reset_seed(fixed_seed)
Random number generators seed has been set to 42
# ModelCheckpoint is used to save the model at certain intervals
checkpoint_grayscale1 = ModelCheckpoint("model1_grayscale.h5", # file path where the model will be saved
monitor='val_accuracy', # means the callback monitors the validation accuracy
verbose=1, # model is saved only when the metric (val_acc) has improved
save_best_only=True,
mode='max')
# Configured callbacks will be passed to the model during training
callbacks_list_grayscale1 = [early_stopping, checkpoint_grayscale1, reduce_learningrate]
# Number of iterations over the entire dataset that the training process should run
epochs = 20
# Building the model
model1_gray = cnn_model_1(1)
# Print first model's summary
model1_gray.summary()
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 48, 48, 64) 320
max_pooling2d (MaxPooling2 (None, 24, 24, 64) 0
D)
dropout (Dropout) (None, 24, 24, 64) 0
conv2d_1 (Conv2D) (None, 24, 24, 32) 8224
max_pooling2d_1 (MaxPoolin (None, 12, 12, 32) 0
g2D)
dropout_1 (Dropout) (None, 12, 12, 32) 0
conv2d_2 (Conv2D) (None, 12, 12, 32) 4128
max_pooling2d_2 (MaxPoolin (None, 6, 6, 32) 0
g2D)
dropout_2 (Dropout) (None, 6, 6, 32) 0
flatten (Flatten) (None, 1152) 0
dense (Dense) (None, 512) 590336
dropout_3 (Dropout) (None, 512) 0
dense_1 (Dense) (None, 4) 2052
=================================================================
Total params: 605060 (2.31 MB)
Trainable params: 605060 (2.31 MB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________
# Fitting the model with required parameters above, saving model into history variable
history_grayscale1 = model1_gray.fit(
train_set_gray,
steps_per_epoch = train_set_gray.samples // train_set_gray.batch_size,
validation_data = validation_set_gray,
validation_steps = validation_set_gray.samples // validation_set_gray.batch_size,
epochs = 20,
verbose = 1,
callbacks = callbacks_list_grayscale1
)
Epoch 1/20 472/472 [==============================] - ETA: 0s - loss: 1.3379 - accuracy: 0.3391 Epoch 1: val_accuracy improved from -inf to 0.46875, saving model to model1_grayscale.h5 472/472 [==============================] - 13s 27ms/step - loss: 1.3379 - accuracy: 0.3391 - val_loss: 1.2111 - val_accuracy: 0.4688 - lr: 0.0010 Epoch 2/20 5/472 [..............................] - ETA: 14s - loss: 1.2863 - accuracy: 0.4000
/Users/ceb/anaconda3/lib/python3.11/site-packages/keras/src/engine/training.py:3103: UserWarning: You are saving your model as an HDF5 file via `model.save()`. This file format is considered legacy. We recommend using instead the native Keras format, e.g. `model.save('my_model.keras')`.
saving_api.save_model(
471/472 [============================>.] - ETA: 0s - loss: 1.2135 - accuracy: 0.4542 Epoch 2: val_accuracy improved from 0.46875 to 0.51552, saving model to model1_grayscale.h5 472/472 [==============================] - 12s 26ms/step - loss: 1.2135 - accuracy: 0.4541 - val_loss: 1.1344 - val_accuracy: 0.5155 - lr: 0.0010 Epoch 3/20 471/472 [============================>.] - ETA: 0s - loss: 1.1304 - accuracy: 0.4964 Epoch 3: val_accuracy improved from 0.51552 to 0.55625, saving model to model1_grayscale.h5 472/472 [==============================] - 13s 27ms/step - loss: 1.1303 - accuracy: 0.4964 - val_loss: 1.0474 - val_accuracy: 0.5562 - lr: 0.0010 Epoch 4/20 472/472 [==============================] - ETA: 0s - loss: 1.0900 - accuracy: 0.5176 Epoch 4: val_accuracy did not improve from 0.55625 472/472 [==============================] - 12s 25ms/step - loss: 1.0900 - accuracy: 0.5176 - val_loss: 1.0598 - val_accuracy: 0.5482 - lr: 0.0010 Epoch 5/20 472/472 [==============================] - ETA: 0s - loss: 1.0431 - accuracy: 0.5460 Epoch 5: val_accuracy improved from 0.55625 to 0.58266, saving model to model1_grayscale.h5 472/472 [==============================] - 12s 25ms/step - loss: 1.0431 - accuracy: 0.5460 - val_loss: 0.9935 - val_accuracy: 0.5827 - lr: 0.0010 Epoch 6/20 471/472 [============================>.] - ETA: 0s - loss: 1.0222 - accuracy: 0.5595 Epoch 6: val_accuracy improved from 0.58266 to 0.59294, saving model to model1_grayscale.h5 472/472 [==============================] - 12s 25ms/step - loss: 1.0220 - accuracy: 0.5596 - val_loss: 0.9528 - val_accuracy: 0.5929 - lr: 0.0010 Epoch 7/20 472/472 [==============================] - ETA: 0s - loss: 1.0013 - accuracy: 0.5666 Epoch 7: val_accuracy improved from 0.59294 to 0.60302, saving model to model1_grayscale.h5 472/472 [==============================] - 12s 25ms/step - loss: 1.0013 - accuracy: 0.5666 - val_loss: 0.9354 - val_accuracy: 0.6030 - lr: 0.0010 Epoch 8/20 470/472 [============================>.] - ETA: 0s - loss: 0.9904 - accuracy: 0.5714 Epoch 8: val_accuracy improved from 0.60302 to 0.62097, saving model to model1_grayscale.h5 472/472 [==============================] - 12s 26ms/step - loss: 0.9907 - accuracy: 0.5716 - val_loss: 0.9110 - val_accuracy: 0.6210 - lr: 0.0010 Epoch 9/20 470/472 [============================>.] - ETA: 0s - loss: 0.9641 - accuracy: 0.5796 Epoch 9: val_accuracy improved from 0.62097 to 0.63044, saving model to model1_grayscale.h5 472/472 [==============================] - 12s 25ms/step - loss: 0.9638 - accuracy: 0.5800 - val_loss: 0.9019 - val_accuracy: 0.6304 - lr: 0.0010 Epoch 10/20 471/472 [============================>.] - ETA: 0s - loss: 0.9505 - accuracy: 0.5922 Epoch 10: val_accuracy did not improve from 0.63044 472/472 [==============================] - 12s 25ms/step - loss: 0.9511 - accuracy: 0.5918 - val_loss: 0.9001 - val_accuracy: 0.6282 - lr: 0.0010 Epoch 11/20 470/472 [============================>.] - ETA: 0s - loss: 0.9368 - accuracy: 0.5989 Epoch 11: val_accuracy did not improve from 0.63044 472/472 [==============================] - 12s 26ms/step - loss: 0.9366 - accuracy: 0.5990 - val_loss: 0.9057 - val_accuracy: 0.6232 - lr: 0.0010 Epoch 12/20 471/472 [============================>.] - ETA: 0s - loss: 0.9254 - accuracy: 0.6050 Epoch 12: val_accuracy improved from 0.63044 to 0.64375, saving model to model1_grayscale.h5 472/472 [==============================] - 12s 26ms/step - loss: 0.9249 - accuracy: 0.6051 - val_loss: 0.8743 - val_accuracy: 0.6438 - lr: 0.0010 Epoch 13/20 470/472 [============================>.] - ETA: 0s - loss: 0.9142 - accuracy: 0.6088 Epoch 13: val_accuracy did not improve from 0.64375 472/472 [==============================] - 12s 26ms/step - loss: 0.9144 - accuracy: 0.6086 - val_loss: 0.8682 - val_accuracy: 0.6371 - lr: 0.0010 Epoch 14/20 472/472 [==============================] - ETA: 0s - loss: 0.9004 - accuracy: 0.6109 Epoch 14: val_accuracy did not improve from 0.64375 472/472 [==============================] - 12s 25ms/step - loss: 0.9004 - accuracy: 0.6109 - val_loss: 0.8820 - val_accuracy: 0.6345 - lr: 0.0010 Epoch 15/20 471/472 [============================>.] - ETA: 0s - loss: 0.8977 - accuracy: 0.6205 Epoch 15: val_accuracy did not improve from 0.64375 472/472 [==============================] - 12s 26ms/step - loss: 0.8980 - accuracy: 0.6203 - val_loss: 0.8789 - val_accuracy: 0.6389 - lr: 0.0010 Epoch 16/20 472/472 [==============================] - ETA: 0s - loss: 0.8903 - accuracy: 0.6174 Epoch 16: val_accuracy improved from 0.64375 to 0.65363, saving model to model1_grayscale.h5 472/472 [==============================] - 12s 26ms/step - loss: 0.8903 - accuracy: 0.6174 - val_loss: 0.8336 - val_accuracy: 0.6536 - lr: 0.0010 Epoch 17/20 471/472 [============================>.] - ETA: 0s - loss: 0.8807 - accuracy: 0.6275 Epoch 17: val_accuracy did not improve from 0.65363 472/472 [==============================] - 12s 26ms/step - loss: 0.8805 - accuracy: 0.6276 - val_loss: 0.8538 - val_accuracy: 0.6456 - lr: 0.0010 Epoch 18/20 472/472 [==============================] - ETA: 0s - loss: 0.8693 - accuracy: 0.6273 Epoch 18: val_accuracy improved from 0.65363 to 0.66331, saving model to model1_grayscale.h5 472/472 [==============================] - 12s 26ms/step - loss: 0.8693 - accuracy: 0.6273 - val_loss: 0.8141 - val_accuracy: 0.6633 - lr: 0.0010 Epoch 19/20 472/472 [==============================] - ETA: 0s - loss: 0.8609 - accuracy: 0.6359 Epoch 19: val_accuracy did not improve from 0.66331 472/472 [==============================] - 13s 27ms/step - loss: 0.8609 - accuracy: 0.6359 - val_loss: 0.8188 - val_accuracy: 0.6569 - lr: 0.0010 Epoch 20/20 472/472 [==============================] - ETA: 0s - loss: 0.8411 - accuracy: 0.6495 Epoch 20: val_accuracy improved from 0.66331 to 0.66573, saving model to model1_grayscale.h5 472/472 [==============================] - 13s 27ms/step - loss: 0.8411 - accuracy: 0.6495 - val_loss: 0.8293 - val_accuracy: 0.6657 - lr: 0.0010
# Load the saved model if notebook session gets disconnected
model1_gray = load_model("model1_grayscale.h5")
# Evaluate the model on the test data using Grayscale
test_loss, test_accuracy = model1_gray.evaluate(test_set_gray, verbose=1)
# Print the results
print("Test Loss:", test_loss)
print("Test Accuracy:", test_accuracy)
4/4 [==============================] - 0s 15ms/step - loss: 0.8388 - accuracy: 0.5938 Test Loss: 0.838821530342102 Test Accuracy: 0.59375
# Plotting the Training and Validation accuracies of CNN model 1 and Grayscale images
plt.plot(history_grayscale1.history['accuracy'])
plt.plot(history_grayscale1.history['val_accuracy'])
plt.title('CNN Model 1 Accuracy with Grayscale Images')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc = 'upper left')
# Display the plot
plt.show()
Observations and Insights:
# Clearing backend
backend.clear_session()
# Reset seed for number generators using reset_seed() function
reset_seed(fixed_seed)
Random number generators seed has been set to 42
# Define more complex CNN model
def cnn_model_2(colour_layers):
cnn_model = Sequential()
# First Convolutional layer with 256 filters and the kernel size of 3x3, 'same' padding and input shape = (48, 48, 3 - RGB colours)
cnn_model.add(Conv2D(filters = 256, kernel_size = (2, 2), padding = "same", input_shape = (img_size[0], img_size[1], colour_layers), activation = 'relu'))
# Adding a BatchNormalization layer
cnn_model.add(BatchNormalization())
# LeakyRelU layer with Leaky ReLU parameter of 0.1
cnn_model.add(LeakyReLU(0.1))
# A max-pooling layer with a pool size of 2x2
cnn_model.add(MaxPooling2D(pool_size = (2, 2)))
# Second Convolutional layer with 128 filters and the kernel size of 2x2 with 'same' padding
cnn_model.add(Conv2D(filters = 128, kernel_size = (2, 2), padding = "same", activation = 'relu'))
# Adding a BatchNormalization layer
cnn_model.add(BatchNormalization())
# LeakyRelU layer with Leaky ReLU parameter of 0.1
cnn_model.add(LeakyReLU(0.1))
# A max-pooling layer with a pool size of 2x2
cnn_model.add(MaxPooling2D(pool_size = (2, 2)))
# Third Convolutional layer with 32 filters and the kernel size of 2x2 with 'same' padding
cnn_model.add(Conv2D(filters = 64, kernel_size = (2, 2), padding = "same", activation = 'relu'))
# Adding a BatchNormalization layer
cnn_model.add(BatchNormalization())
# LeakyRelU layer with Leaky ReLU parameter of 0.1
cnn_model.add(LeakyReLU(0.1))
# A max-pooling layer with a pool size of 2x2
cnn_model.add(MaxPooling2D(pool_size = (2, 2)))
# Fourth Convolutional layer with 32 filters and the kernel size of 2x2 with 'same' padding
cnn_model.add(Conv2D(filters = 32, kernel_size = (2, 2), padding = "same", activation = 'relu'))
# Adding a BatchNormalization layer
cnn_model.add(BatchNormalization())
# LeakyRelU layer with Leaky ReLU parameter of 0.1
cnn_model.add(LeakyReLU(0.1))
# A max-pooling layer with a pool size of 2x2
cnn_model.add(MaxPooling2D(pool_size = (2, 2)))
# Flatten the output from the previous layer
cnn_model.add(Flatten())
# Dense layer with 512 nodes
cnn_model.add(Dense(512, activation = 'relu'))
# Dense layer with 128 nodes
cnn_model.add(Dense(128, activation = 'relu'))
# Dropout layer with the rate equal to 0.4
cnn_model.add(Dropout(0.4))
# Final output layer with nodes equal to the number of classes (sad, happy, neutral and surprised) and 'softmax' as the activation function
cnn_model.add(Dense(4, activation = 'softmax'))
# Compiling the CNN model with categorical crossentropy as loss function, Adam Optimizer
# with 0.001 learning rate, and set metrics set to 'accuracy'.
cnn_model.compile(loss='categorical_crossentropy',
optimizer=legacy.Adam(learning_rate = 0.001),
metrics=['accuracy'])
return cnn_model
# Building the model
model2_rgb = cnn_model_2(3)
# Print first model's summary
model2_rgb.summary()
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 48, 48, 256) 3328
batch_normalization (Batch (None, 48, 48, 256) 1024
Normalization)
leaky_re_lu (LeakyReLU) (None, 48, 48, 256) 0
max_pooling2d (MaxPooling2 (None, 24, 24, 256) 0
D)
conv2d_1 (Conv2D) (None, 24, 24, 128) 131200
batch_normalization_1 (Bat (None, 24, 24, 128) 512
chNormalization)
leaky_re_lu_1 (LeakyReLU) (None, 24, 24, 128) 0
max_pooling2d_1 (MaxPoolin (None, 12, 12, 128) 0
g2D)
conv2d_2 (Conv2D) (None, 12, 12, 64) 32832
batch_normalization_2 (Bat (None, 12, 12, 64) 256
chNormalization)
leaky_re_lu_2 (LeakyReLU) (None, 12, 12, 64) 0
max_pooling2d_2 (MaxPoolin (None, 6, 6, 64) 0
g2D)
conv2d_3 (Conv2D) (None, 6, 6, 32) 8224
batch_normalization_3 (Bat (None, 6, 6, 32) 128
chNormalization)
leaky_re_lu_3 (LeakyReLU) (None, 6, 6, 32) 0
max_pooling2d_3 (MaxPoolin (None, 3, 3, 32) 0
g2D)
flatten (Flatten) (None, 288) 0
dense (Dense) (None, 512) 147968
dense_1 (Dense) (None, 128) 65664
dropout (Dropout) (None, 128) 0
dense_2 (Dense) (None, 4) 516
=================================================================
Total params: 391652 (1.49 MB)
Trainable params: 390692 (1.49 MB)
Non-trainable params: 960 (3.75 KB)
_________________________________________________________________
# Visualize the model as blocks
plot_model(model2_rgb, to_file='model.png', show_shapes=True, show_layer_names=True)
# ModelCheckpoint is used to save the model at certain intervals
checkpoint_model2_rgb = ModelCheckpoint("model2_rgb.h5", # file path where the model will be saved
monitor='val_accuracy', # means the callback monitors the validation accuracy
verbose=1, # model is saved only when the metric (val_acc) has improved
save_best_only=True,
mode='max')
# Configured callbacks will be passed to the model during training
callbacks_list_model2_rgb = [early_stopping, checkpoint_model2_rgb, reduce_learningrate]
# Number of iterations over the entire dataset that the training process should run
epochs = 20
# Fitting the model with required parameters above, saving model into history_model2 variable
history_model2_rgb = model2_rgb.fit(
train_set_rgb,
steps_per_epoch = train_set_rgb.samples // train_set_rgb.batch_size,
validation_data = validation_set_rgb,
validation_steps = validation_set_rgb.samples // validation_set_rgb.batch_size,
epochs = 20,
verbose = 1,
callbacks = callbacks_list_model2_rgb
)
Epoch 1/20 472/472 [==============================] - ETA: 0s - loss: 1.3467 - accuracy: 0.3299 Epoch 1: val_accuracy improved from -inf to 0.37036, saving model to model2_rgb.h5 472/472 [==============================] - 83s 175ms/step - loss: 1.3467 - accuracy: 0.3299 - val_loss: 1.3109 - val_accuracy: 0.3704 - lr: 0.0010 Epoch 2/20 472/472 [==============================] - ETA: 0s - loss: 1.2211 - accuracy: 0.4125 Epoch 2: val_accuracy improved from 0.37036 to 0.42702, saving model to model2_rgb.h5 472/472 [==============================] - 81s 172ms/step - loss: 1.2211 - accuracy: 0.4125 - val_loss: 1.1961 - val_accuracy: 0.4270 - lr: 0.0010 Epoch 3/20 472/472 [==============================] - ETA: 0s - loss: 1.1291 - accuracy: 0.4863 Epoch 3: val_accuracy improved from 0.42702 to 0.47520, saving model to model2_rgb.h5 472/472 [==============================] - 82s 173ms/step - loss: 1.1291 - accuracy: 0.4863 - val_loss: 1.1530 - val_accuracy: 0.4752 - lr: 0.0010 Epoch 4/20 472/472 [==============================] - ETA: 0s - loss: 1.0326 - accuracy: 0.5530 Epoch 4: val_accuracy improved from 0.47520 to 0.55867, saving model to model2_rgb.h5 472/472 [==============================] - 82s 173ms/step - loss: 1.0326 - accuracy: 0.5530 - val_loss: 1.0323 - val_accuracy: 0.5587 - lr: 0.0010 Epoch 5/20 472/472 [==============================] - ETA: 0s - loss: 0.9555 - accuracy: 0.5907 Epoch 5: val_accuracy improved from 0.55867 to 0.60565, saving model to model2_rgb.h5 472/472 [==============================] - 82s 173ms/step - loss: 0.9555 - accuracy: 0.5907 - val_loss: 0.9409 - val_accuracy: 0.6056 - lr: 0.0010 Epoch 6/20 472/472 [==============================] - ETA: 0s - loss: 0.9048 - accuracy: 0.6131 Epoch 6: val_accuracy did not improve from 0.60565 472/472 [==============================] - 81s 172ms/step - loss: 0.9048 - accuracy: 0.6131 - val_loss: 0.9495 - val_accuracy: 0.5954 - lr: 0.0010 Epoch 7/20 472/472 [==============================] - ETA: 0s - loss: 0.8587 - accuracy: 0.6407 Epoch 7: val_accuracy improved from 0.60565 to 0.61935, saving model to model2_rgb.h5 472/472 [==============================] - 82s 174ms/step - loss: 0.8587 - accuracy: 0.6407 - val_loss: 0.9314 - val_accuracy: 0.6194 - lr: 0.0010 Epoch 8/20 472/472 [==============================] - ETA: 0s - loss: 0.8298 - accuracy: 0.6519 Epoch 8: val_accuracy improved from 0.61935 to 0.62883, saving model to model2_rgb.h5 472/472 [==============================] - 84s 177ms/step - loss: 0.8298 - accuracy: 0.6519 - val_loss: 0.8953 - val_accuracy: 0.6288 - lr: 0.0010 Epoch 9/20 472/472 [==============================] - ETA: 0s - loss: 0.8065 - accuracy: 0.6612 Epoch 9: val_accuracy improved from 0.62883 to 0.65000, saving model to model2_rgb.h5 472/472 [==============================] - 83s 175ms/step - loss: 0.8065 - accuracy: 0.6612 - val_loss: 0.8580 - val_accuracy: 0.6500 - lr: 0.0010 Epoch 10/20 472/472 [==============================] - ETA: 0s - loss: 0.7860 - accuracy: 0.6741 Epoch 10: val_accuracy improved from 0.65000 to 0.65887, saving model to model2_rgb.h5 472/472 [==============================] - 82s 174ms/step - loss: 0.7860 - accuracy: 0.6741 - val_loss: 0.8170 - val_accuracy: 0.6589 - lr: 0.0010 Epoch 11/20 472/472 [==============================] - ETA: 0s - loss: 0.7630 - accuracy: 0.6815 Epoch 11: val_accuracy improved from 0.65887 to 0.66512, saving model to model2_rgb.h5 472/472 [==============================] - 81s 172ms/step - loss: 0.7630 - accuracy: 0.6815 - val_loss: 0.8163 - val_accuracy: 0.6651 - lr: 0.0010 Epoch 12/20 472/472 [==============================] - ETA: 0s - loss: 0.7426 - accuracy: 0.6910 Epoch 12: val_accuracy did not improve from 0.66512 472/472 [==============================] - 82s 174ms/step - loss: 0.7426 - accuracy: 0.6910 - val_loss: 0.8572 - val_accuracy: 0.6490 - lr: 0.0010 Epoch 13/20 472/472 [==============================] - ETA: 0s - loss: 0.7290 - accuracy: 0.6964 Epoch 13: val_accuracy improved from 0.66512 to 0.68185, saving model to model2_rgb.h5 472/472 [==============================] - 83s 176ms/step - loss: 0.7290 - accuracy: 0.6964 - val_loss: 0.7796 - val_accuracy: 0.6819 - lr: 0.0010 Epoch 14/20 472/472 [==============================] - ETA: 0s - loss: 0.7147 - accuracy: 0.7033 Epoch 14: val_accuracy did not improve from 0.68185 472/472 [==============================] - 84s 177ms/step - loss: 0.7147 - accuracy: 0.7033 - val_loss: 0.8151 - val_accuracy: 0.6629 - lr: 0.0010 Epoch 15/20 472/472 [==============================] - ETA: 0s - loss: 0.7002 - accuracy: 0.7074 Epoch 15: val_accuracy did not improve from 0.68185 472/472 [==============================] - 82s 175ms/step - loss: 0.7002 - accuracy: 0.7074 - val_loss: 0.8035 - val_accuracy: 0.6746 - lr: 0.0010 Epoch 16/20 472/472 [==============================] - ETA: 0s - loss: 0.6898 - accuracy: 0.7120Restoring model weights from the end of the best epoch: 13. Epoch 16: val_accuracy did not improve from 0.68185 Epoch 16: ReduceLROnPlateau reducing learning rate to 0.00020000000949949026. 472/472 [==============================] - 83s 175ms/step - loss: 0.6898 - accuracy: 0.7120 - val_loss: 0.7956 - val_accuracy: 0.6802 - lr: 0.0010 Epoch 16: early stopping
# Load the saved model if notebook session gets disconnected
model2_rgb = load_model("model2_rgb.h5")
# Evaluate the model on the test data using RGB model
test_loss, test_accuracy = model2_rgb.evaluate(test_set_rgb, verbose=1)
# Print the results
print("Test Loss:", test_loss)
print("Test Accuracy:", test_accuracy)
4/4 [==============================] - 0s 45ms/step - loss: 0.7281 - accuracy: 0.6719 Test Loss: 0.728143572807312 Test Accuracy: 0.671875
# Plotting the Training and Validation accuracies of CNN model 2 and RGB images
plt.plot(history_model2_rgb.history['accuracy'])
plt.plot(history_model2_rgb.history['val_accuracy'])
plt.title('CNN Model 2 Accuracy with RGB Images')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc = 'upper left')
# Display the plot
plt.show()
# Clearing backend
backend.clear_session()
# Reset seed for number generators using reset_seed() function
reset_seed(fixed_seed)
Random number generators seed has been set to 42
# ModelCheckpoint is used to save the model at certain intervals
checkpoint_model2_gray = ModelCheckpoint("model2_grayscale.h5", # file path where the model will be saved
monitor='val_accuracy', # means the callback monitors the validation accuracy
verbose=1, # model is saved only when the metric (val_acc) has improved
save_best_only=True,
mode='max')
# Configured callbacks will be passed to the model during training
callbacks_list_model2_gray = [early_stopping, checkpoint_model2_gray, reduce_learningrate]
# Number of iterations over the entire dataset that the training process should run
epochs = 20
# Building the model
model2_gray = cnn_model_2(1)
# Print first model's summary
model2_gray.summary()
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 48, 48, 256) 1280
batch_normalization (Batch (None, 48, 48, 256) 1024
Normalization)
leaky_re_lu (LeakyReLU) (None, 48, 48, 256) 0
max_pooling2d (MaxPooling2 (None, 24, 24, 256) 0
D)
conv2d_1 (Conv2D) (None, 24, 24, 128) 131200
batch_normalization_1 (Bat (None, 24, 24, 128) 512
chNormalization)
leaky_re_lu_1 (LeakyReLU) (None, 24, 24, 128) 0
max_pooling2d_1 (MaxPoolin (None, 12, 12, 128) 0
g2D)
conv2d_2 (Conv2D) (None, 12, 12, 64) 32832
batch_normalization_2 (Bat (None, 12, 12, 64) 256
chNormalization)
leaky_re_lu_2 (LeakyReLU) (None, 12, 12, 64) 0
max_pooling2d_2 (MaxPoolin (None, 6, 6, 64) 0
g2D)
conv2d_3 (Conv2D) (None, 6, 6, 32) 8224
batch_normalization_3 (Bat (None, 6, 6, 32) 128
chNormalization)
leaky_re_lu_3 (LeakyReLU) (None, 6, 6, 32) 0
max_pooling2d_3 (MaxPoolin (None, 3, 3, 32) 0
g2D)
flatten (Flatten) (None, 288) 0
dense (Dense) (None, 512) 147968
dense_1 (Dense) (None, 128) 65664
dropout (Dropout) (None, 128) 0
dense_2 (Dense) (None, 4) 516
=================================================================
Total params: 389604 (1.49 MB)
Trainable params: 388644 (1.48 MB)
Non-trainable params: 960 (3.75 KB)
_________________________________________________________________
# Fitting the model with required parameters above, saving model into history_model2_gray variable
history_model2_gray = model2_gray.fit(
train_set_gray,
steps_per_epoch = train_set_gray.samples // train_set_gray.batch_size,
validation_data = validation_set_gray,
validation_steps = validation_set_gray.samples // validation_set_gray.batch_size,
epochs = 20,
verbose = 1,
callbacks = callbacks_list_model2_gray
)
Epoch 1/20 472/472 [==============================] - ETA: 0s - loss: 1.3722 - accuracy: 0.3058 Epoch 1: val_accuracy improved from -inf to 0.39980, saving model to model2_grayscale.h5 472/472 [==============================] - 85s 180ms/step - loss: 1.3722 - accuracy: 0.3058 - val_loss: 1.3364 - val_accuracy: 0.3998 - lr: 0.0010 Epoch 2/20 472/472 [==============================] - ETA: 0s - loss: 1.2458 - accuracy: 0.3947 Epoch 2: val_accuracy improved from 0.39980 to 0.44214, saving model to model2_grayscale.h5 472/472 [==============================] - 86s 182ms/step - loss: 1.2458 - accuracy: 0.3947 - val_loss: 1.2179 - val_accuracy: 0.4421 - lr: 0.0010 Epoch 3/20 472/472 [==============================] - ETA: 0s - loss: 1.1441 - accuracy: 0.4674 Epoch 3: val_accuracy improved from 0.44214 to 0.44738, saving model to model2_grayscale.h5 472/472 [==============================] - 86s 182ms/step - loss: 1.1441 - accuracy: 0.4674 - val_loss: 1.1624 - val_accuracy: 0.4474 - lr: 0.0010 Epoch 4/20 472/472 [==============================] - ETA: 0s - loss: 1.0328 - accuracy: 0.5486 Epoch 4: val_accuracy improved from 0.44738 to 0.54536, saving model to model2_grayscale.h5 472/472 [==============================] - 86s 183ms/step - loss: 1.0328 - accuracy: 0.5486 - val_loss: 1.0824 - val_accuracy: 0.5454 - lr: 0.0010 Epoch 5/20 472/472 [==============================] - ETA: 0s - loss: 0.9544 - accuracy: 0.5885 Epoch 5: val_accuracy improved from 0.54536 to 0.60786, saving model to model2_grayscale.h5 472/472 [==============================] - 87s 185ms/step - loss: 0.9544 - accuracy: 0.5885 - val_loss: 0.9520 - val_accuracy: 0.6079 - lr: 0.0010 Epoch 6/20 472/472 [==============================] - ETA: 0s - loss: 0.9004 - accuracy: 0.6175 Epoch 6: val_accuracy improved from 0.60786 to 0.63710, saving model to model2_grayscale.h5 472/472 [==============================] - 85s 180ms/step - loss: 0.9004 - accuracy: 0.6175 - val_loss: 0.8746 - val_accuracy: 0.6371 - lr: 0.0010 Epoch 7/20 472/472 [==============================] - ETA: 0s - loss: 0.8636 - accuracy: 0.6322 Epoch 7: val_accuracy did not improve from 0.63710 472/472 [==============================] - 87s 183ms/step - loss: 0.8636 - accuracy: 0.6322 - val_loss: 0.9433 - val_accuracy: 0.6095 - lr: 0.0010 Epoch 8/20 472/472 [==============================] - ETA: 0s - loss: 0.8300 - accuracy: 0.6528 Epoch 8: val_accuracy improved from 0.63710 to 0.65464, saving model to model2_grayscale.h5 472/472 [==============================] - 86s 182ms/step - loss: 0.8300 - accuracy: 0.6528 - val_loss: 0.8286 - val_accuracy: 0.6546 - lr: 0.0010 Epoch 9/20 472/472 [==============================] - ETA: 0s - loss: 0.8099 - accuracy: 0.6596 Epoch 9: val_accuracy improved from 0.65464 to 0.65988, saving model to model2_grayscale.h5 472/472 [==============================] - 86s 183ms/step - loss: 0.8099 - accuracy: 0.6596 - val_loss: 0.8221 - val_accuracy: 0.6599 - lr: 0.0010 Epoch 10/20 472/472 [==============================] - ETA: 0s - loss: 0.7813 - accuracy: 0.6725 Epoch 10: val_accuracy did not improve from 0.65988 472/472 [==============================] - 84s 177ms/step - loss: 0.7813 - accuracy: 0.6725 - val_loss: 0.8526 - val_accuracy: 0.6470 - lr: 0.0010 Epoch 11/20 472/472 [==============================] - ETA: 0s - loss: 0.7628 - accuracy: 0.6816 Epoch 11: val_accuracy improved from 0.65988 to 0.66794, saving model to model2_grayscale.h5 472/472 [==============================] - 84s 178ms/step - loss: 0.7628 - accuracy: 0.6816 - val_loss: 0.8061 - val_accuracy: 0.6679 - lr: 0.0010 Epoch 12/20 472/472 [==============================] - ETA: 0s - loss: 0.7431 - accuracy: 0.6893 Epoch 12: val_accuracy did not improve from 0.66794 472/472 [==============================] - 84s 178ms/step - loss: 0.7431 - accuracy: 0.6893 - val_loss: 0.8002 - val_accuracy: 0.6677 - lr: 0.0010 Epoch 13/20 472/472 [==============================] - ETA: 0s - loss: 0.7252 - accuracy: 0.6987 Epoch 13: val_accuracy did not improve from 0.66794 472/472 [==============================] - 84s 179ms/step - loss: 0.7252 - accuracy: 0.6987 - val_loss: 0.8336 - val_accuracy: 0.6639 - lr: 0.0010 Epoch 14/20 472/472 [==============================] - ETA: 0s - loss: 0.7122 - accuracy: 0.7021 Epoch 14: val_accuracy did not improve from 0.66794 472/472 [==============================] - 83s 175ms/step - loss: 0.7122 - accuracy: 0.7021 - val_loss: 0.8503 - val_accuracy: 0.6456 - lr: 0.0010 Epoch 15/20 472/472 [==============================] - ETA: 0s - loss: 0.6940 - accuracy: 0.7109Restoring model weights from the end of the best epoch: 12. Epoch 15: val_accuracy improved from 0.66794 to 0.67984, saving model to model2_grayscale.h5 Epoch 15: ReduceLROnPlateau reducing learning rate to 0.00020000000949949026. 472/472 [==============================] - 84s 178ms/step - loss: 0.6940 - accuracy: 0.7109 - val_loss: 0.8075 - val_accuracy: 0.6798 - lr: 0.0010 Epoch 15: early stopping
# Load the saved model if notebook session gets disconnected
model2_gray = load_model("model2_grayscale.h5")
# Evaluate the model on the test data using Grayscale model
test_loss, test_accuracy = model2_gray.evaluate(test_set_gray, verbose=1)
# Print the results
print("Test Loss:", test_loss)
print("Test Accuracy:", test_accuracy)
4/4 [==============================] - 0s 40ms/step - loss: 0.7769 - accuracy: 0.6250 Test Loss: 0.7768638134002686 Test Accuracy: 0.625
# Plotting the Training and Validation accuracies of CNN model 2 and Grayscale images
plt.plot(history_model2_gray.history['accuracy'])
plt.plot(history_model2_gray.history['val_accuracy'])
plt.title('CNN Model 2 Accuracy with Grayscale Images')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc = 'upper left')
# Display the plot
plt.show()
Observations and Insights:
In this section, we will create several Transfer Learning architectures. For the pre-trained models, we will select three popular architectures namely, VGG16, ResNet v2, and Efficient Net. The difference between these architectures and the previous architectures is that these will require 3 input channels while the earlier ones worked on 'grayscale' images. Therefore, we need to create new DataLoaders.
In this section, we are creating data loaders that we will use as inputs to our Neural Network. We will have to go with color_mode = 'rgb' as this is the required format for the transfer learning architectures.
# Creating dataloader for train for RGB dataset
train_set_pre = datagen.flow_from_directory(folder_path + "train",
target_size = (img_size),
color_mode = 'rgb',
batch_size = batch_size,
class_mode = 'categorical',
classes = ['happy', 'sad', 'neutral', 'surprise'],
shuffle = True)
# Creating dataloader for validation for RGB dataset
validation_set_pre = datagen.flow_from_directory(folder_path + "validation",
target_size = (img_size),
color_mode = 'rgb',
batch_size = batch_size,
class_mode = 'categorical',
classes = ['happy', 'sad', 'neutral', 'surprise'],
shuffle = True)
# Creating dataloader for test for RGB dataset
test_set_pre = datagen.flow_from_directory(folder_path + "test",
target_size = (img_size),
color_mode = 'rgb',
batch_size = batch_size,
class_mode = 'categorical',
classes = ['happy', 'sad', 'neutral', 'surprise'],
shuffle = True)
Found 15109 images belonging to 4 classes. Found 4977 images belonging to 4 classes. Found 128 images belonging to 4 classes.
# Clearing backend
backend.clear_session()
# Reset seed for number generators using reset_seed() function
reset_seed(fixed_seed)
Random number generators seed has been set to 42
from keras.applications import VGG16
from keras.models import Model
from keras.layers import Flatten, Dense, Dropout, BatchNormalization, Input
# Load the VGG16 model without top layers
vgg_base = VGG16(include_top=False,
weights='imagenet',
input_tensor=Input(shape=(img_size[0],img_size[1], 3)))
# Set the VGG16 model to be non-trainable
vgg_base.trainable = False
# Rebuild the model
x = vgg_base.output
x = Flatten()(x)
x = Dense(256, activation='relu')(x)
x = Dense(128, activation='relu')(x)
x = Dropout(0.3)(x)
x = Dense(64, activation='relu')(x)
x = BatchNormalization()(x)
pred = Dense(4, activation='softmax')(x)
# Create a new model
vggmodel = Model(inputs=vgg_base.input, outputs=pred)
# Print the model summary
vggmodel.summary()
Model: "model"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) [(None, 48, 48, 3)] 0
block1_conv1 (Conv2D) (None, 48, 48, 64) 1792
block1_conv2 (Conv2D) (None, 48, 48, 64) 36928
block1_pool (MaxPooling2D) (None, 24, 24, 64) 0
block2_conv1 (Conv2D) (None, 24, 24, 128) 73856
block2_conv2 (Conv2D) (None, 24, 24, 128) 147584
block2_pool (MaxPooling2D) (None, 12, 12, 128) 0
block3_conv1 (Conv2D) (None, 12, 12, 256) 295168
block3_conv2 (Conv2D) (None, 12, 12, 256) 590080
block3_conv3 (Conv2D) (None, 12, 12, 256) 590080
block3_pool (MaxPooling2D) (None, 6, 6, 256) 0
block4_conv1 (Conv2D) (None, 6, 6, 512) 1180160
block4_conv2 (Conv2D) (None, 6, 6, 512) 2359808
block4_conv3 (Conv2D) (None, 6, 6, 512) 2359808
block4_pool (MaxPooling2D) (None, 3, 3, 512) 0
block5_conv1 (Conv2D) (None, 3, 3, 512) 2359808
block5_conv2 (Conv2D) (None, 3, 3, 512) 2359808
block5_conv3 (Conv2D) (None, 3, 3, 512) 2359808
block5_pool (MaxPooling2D) (None, 1, 1, 512) 0
flatten (Flatten) (None, 512) 0
dense (Dense) (None, 256) 131328
dense_1 (Dense) (None, 128) 32896
dropout (Dropout) (None, 128) 0
dense_2 (Dense) (None, 64) 8256
batch_normalization (Batch (None, 64) 256
Normalization)
dense_3 (Dense) (None, 4) 260
=================================================================
Total params: 14887684 (56.79 MB)
Trainable params: 172868 (675.27 KB)
Non-trainable params: 14714816 (56.13 MB)
_________________________________________________________________
# Visualize the model as blocks
plot_model(vggmodel, to_file='model.png', show_shapes=True, show_layer_names=True)
# ModelCheckpoint is used to save the model at certain intervals
checkpoint_vgg = ModelCheckpoint("vggmodel.h5", # file path where the model will be saved
monitor='val_accuracy', # means the callback monitors the validation accuracy
verbose=1, # model is saved only when the metric (val_acc) has improved
save_best_only=True,
mode='max')
# Configured callbacks will be passed to the model during training
callbacks_list_vgg = [early_stopping, checkpoint_vgg, reduce_learningrate]
# Number of iterations over the entire dataset that the training process should run
epochs = 20
# Compiling the CNN model with categorical crossentropy as loss function, Adam Optimizer
# with 0.001 learning rate, and set metrics set to 'accuracy'.
vggmodel.compile(loss='categorical_crossentropy',
optimizer=legacy.Adam(learning_rate = 0.001),
metrics=['accuracy'])
# Fitting the model with required parameters above, saving model into history_vgg variable
history_vgg = vggmodel.fit(
train_set_pre,
steps_per_epoch = train_set_pre.samples // train_set_pre.batch_size,
validation_data = validation_set_pre,
validation_steps = validation_set_pre.samples // validation_set_pre.batch_size,
epochs = 20,
verbose = 1,
callbacks = callbacks_list_vgg
)
Epoch 1/20 472/472 [==============================] - ETA: 0s - loss: 1.3230 - accuracy: 0.3808 Epoch 1: val_accuracy improved from -inf to 0.41250, saving model to vggmodel.h5 472/472 [==============================] - 110s 233ms/step - loss: 1.3230 - accuracy: 0.3808 - val_loss: 1.2388 - val_accuracy: 0.4125 - lr: 0.0010 Epoch 2/20
/Users/ceb/anaconda3/lib/python3.11/site-packages/keras/src/engine/training.py:3103: UserWarning: You are saving your model as an HDF5 file via `model.save()`. This file format is considered legacy. We recommend using instead the native Keras format, e.g. `model.save('my_model.keras')`.
saving_api.save_model(
472/472 [==============================] - ETA: 0s - loss: 1.2147 - accuracy: 0.4441 Epoch 2: val_accuracy improved from 0.41250 to 0.48629, saving model to vggmodel.h5 472/472 [==============================] - 104s 219ms/step - loss: 1.2147 - accuracy: 0.4441 - val_loss: 1.1530 - val_accuracy: 0.4863 - lr: 0.0010 Epoch 3/20 472/472 [==============================] - ETA: 0s - loss: 1.1849 - accuracy: 0.4615 Epoch 3: val_accuracy improved from 0.48629 to 0.50121, saving model to vggmodel.h5 472/472 [==============================] - 95s 202ms/step - loss: 1.1849 - accuracy: 0.4615 - val_loss: 1.1402 - val_accuracy: 0.5012 - lr: 0.0010 Epoch 4/20 472/472 [==============================] - ETA: 0s - loss: 1.1698 - accuracy: 0.4722 Epoch 4: val_accuracy did not improve from 0.50121 472/472 [==============================] - 97s 206ms/step - loss: 1.1698 - accuracy: 0.4722 - val_loss: 1.1603 - val_accuracy: 0.4645 - lr: 0.0010 Epoch 5/20 472/472 [==============================] - ETA: 0s - loss: 1.1534 - accuracy: 0.4791 Epoch 5: val_accuracy did not improve from 0.50121 472/472 [==============================] - 97s 206ms/step - loss: 1.1534 - accuracy: 0.4791 - val_loss: 1.1310 - val_accuracy: 0.4968 - lr: 0.0010 Epoch 6/20 472/472 [==============================] - ETA: 0s - loss: 1.1497 - accuracy: 0.4814 Epoch 6: val_accuracy did not improve from 0.50121 472/472 [==============================] - 98s 206ms/step - loss: 1.1497 - accuracy: 0.4814 - val_loss: 1.1381 - val_accuracy: 0.4875 - lr: 0.0010 Epoch 7/20 472/472 [==============================] - ETA: 0s - loss: 1.1446 - accuracy: 0.4884 Epoch 7: val_accuracy improved from 0.50121 to 0.50464, saving model to vggmodel.h5 472/472 [==============================] - 97s 205ms/step - loss: 1.1446 - accuracy: 0.4884 - val_loss: 1.1194 - val_accuracy: 0.5046 - lr: 0.0010 Epoch 8/20 472/472 [==============================] - ETA: 0s - loss: 1.1359 - accuracy: 0.4970 Epoch 8: val_accuracy did not improve from 0.50464 472/472 [==============================] - 99s 211ms/step - loss: 1.1359 - accuracy: 0.4970 - val_loss: 1.1455 - val_accuracy: 0.4804 - lr: 0.0010 Epoch 9/20 472/472 [==============================] - ETA: 0s - loss: 1.1270 - accuracy: 0.5039 Epoch 9: val_accuracy did not improve from 0.50464 472/472 [==============================] - 96s 204ms/step - loss: 1.1270 - accuracy: 0.5039 - val_loss: 1.1971 - val_accuracy: 0.4681 - lr: 0.0010 Epoch 10/20 472/472 [==============================] - ETA: 0s - loss: 1.1223 - accuracy: 0.5020Restoring model weights from the end of the best epoch: 7. Epoch 10: val_accuracy did not improve from 0.50464 Epoch 10: ReduceLROnPlateau reducing learning rate to 0.00020000000949949026. 472/472 [==============================] - 99s 210ms/step - loss: 1.1223 - accuracy: 0.5020 - val_loss: 1.1743 - val_accuracy: 0.4744 - lr: 0.0010 Epoch 10: early stopping
# Load the saved model if notebook session gets disconnected
vggmodel = load_model("vggmodel.h5")
# Evaluate the model on the test data
test_loss, test_accuracy = vggmodel.evaluate(test_set_pre, verbose=1)
# Print the results
print("Test Loss:", test_loss)
print("Test Accuracy:", test_accuracy)
4/4 [==============================] - 1s 158ms/step - loss: 1.2255 - accuracy: 0.4453 Test Loss: 1.225456714630127 Test Accuracy: 0.4453125
# Plotting the Training and Validation accuracies of VGG16
plt.plot(history_vgg.history['accuracy'])
plt.plot(history_vgg.history['val_accuracy'])
plt.title('VGG16 Transfer Learning Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc = 'upper left')
# Display the plot
plt.show()
Observations and Insights:
# Clearing backend
backend.clear_session()
# Reset seed for number generators using reset_seed() function
reset_seed(fixed_seed)
Random number generators seed has been set to 42
import tensorflow as tf
from tensorflow.keras.applications import ResNet50
from tensorflow.keras.layers import Flatten, Dense, GlobalAveragePooling2D
# Load the ResNet50 model without its top layers and with pre-trained ImageNet weights
Resnet = ResNet50(include_top=False, weights='imagenet', input_tensor=tf.keras.Input(shape=(img_size[0], img_size[1], 3)))
# Print imported model's summary
Resnet.summary()
Model: "resnet50"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_1 (InputLayer) [(None, 48, 48, 3)] 0 []
conv1_pad (ZeroPadding2D) (None, 54, 54, 3) 0 ['input_1[0][0]']
conv1_conv (Conv2D) (None, 24, 24, 64) 9472 ['conv1_pad[0][0]']
conv1_bn (BatchNormalizati (None, 24, 24, 64) 256 ['conv1_conv[0][0]']
on)
conv1_relu (Activation) (None, 24, 24, 64) 0 ['conv1_bn[0][0]']
pool1_pad (ZeroPadding2D) (None, 26, 26, 64) 0 ['conv1_relu[0][0]']
pool1_pool (MaxPooling2D) (None, 12, 12, 64) 0 ['pool1_pad[0][0]']
conv2_block1_1_conv (Conv2 (None, 12, 12, 64) 4160 ['pool1_pool[0][0]']
D)
conv2_block1_1_bn (BatchNo (None, 12, 12, 64) 256 ['conv2_block1_1_conv[0][0]']
rmalization)
conv2_block1_1_relu (Activ (None, 12, 12, 64) 0 ['conv2_block1_1_bn[0][0]']
ation)
conv2_block1_2_conv (Conv2 (None, 12, 12, 64) 36928 ['conv2_block1_1_relu[0][0]']
D)
conv2_block1_2_bn (BatchNo (None, 12, 12, 64) 256 ['conv2_block1_2_conv[0][0]']
rmalization)
conv2_block1_2_relu (Activ (None, 12, 12, 64) 0 ['conv2_block1_2_bn[0][0]']
ation)
conv2_block1_0_conv (Conv2 (None, 12, 12, 256) 16640 ['pool1_pool[0][0]']
D)
conv2_block1_3_conv (Conv2 (None, 12, 12, 256) 16640 ['conv2_block1_2_relu[0][0]']
D)
conv2_block1_0_bn (BatchNo (None, 12, 12, 256) 1024 ['conv2_block1_0_conv[0][0]']
rmalization)
conv2_block1_3_bn (BatchNo (None, 12, 12, 256) 1024 ['conv2_block1_3_conv[0][0]']
rmalization)
conv2_block1_add (Add) (None, 12, 12, 256) 0 ['conv2_block1_0_bn[0][0]',
'conv2_block1_3_bn[0][0]']
conv2_block1_out (Activati (None, 12, 12, 256) 0 ['conv2_block1_add[0][0]']
on)
conv2_block2_1_conv (Conv2 (None, 12, 12, 64) 16448 ['conv2_block1_out[0][0]']
D)
conv2_block2_1_bn (BatchNo (None, 12, 12, 64) 256 ['conv2_block2_1_conv[0][0]']
rmalization)
conv2_block2_1_relu (Activ (None, 12, 12, 64) 0 ['conv2_block2_1_bn[0][0]']
ation)
conv2_block2_2_conv (Conv2 (None, 12, 12, 64) 36928 ['conv2_block2_1_relu[0][0]']
D)
conv2_block2_2_bn (BatchNo (None, 12, 12, 64) 256 ['conv2_block2_2_conv[0][0]']
rmalization)
conv2_block2_2_relu (Activ (None, 12, 12, 64) 0 ['conv2_block2_2_bn[0][0]']
ation)
conv2_block2_3_conv (Conv2 (None, 12, 12, 256) 16640 ['conv2_block2_2_relu[0][0]']
D)
conv2_block2_3_bn (BatchNo (None, 12, 12, 256) 1024 ['conv2_block2_3_conv[0][0]']
rmalization)
conv2_block2_add (Add) (None, 12, 12, 256) 0 ['conv2_block1_out[0][0]',
'conv2_block2_3_bn[0][0]']
conv2_block2_out (Activati (None, 12, 12, 256) 0 ['conv2_block2_add[0][0]']
on)
conv2_block3_1_conv (Conv2 (None, 12, 12, 64) 16448 ['conv2_block2_out[0][0]']
D)
conv2_block3_1_bn (BatchNo (None, 12, 12, 64) 256 ['conv2_block3_1_conv[0][0]']
rmalization)
conv2_block3_1_relu (Activ (None, 12, 12, 64) 0 ['conv2_block3_1_bn[0][0]']
ation)
conv2_block3_2_conv (Conv2 (None, 12, 12, 64) 36928 ['conv2_block3_1_relu[0][0]']
D)
conv2_block3_2_bn (BatchNo (None, 12, 12, 64) 256 ['conv2_block3_2_conv[0][0]']
rmalization)
conv2_block3_2_relu (Activ (None, 12, 12, 64) 0 ['conv2_block3_2_bn[0][0]']
ation)
conv2_block3_3_conv (Conv2 (None, 12, 12, 256) 16640 ['conv2_block3_2_relu[0][0]']
D)
conv2_block3_3_bn (BatchNo (None, 12, 12, 256) 1024 ['conv2_block3_3_conv[0][0]']
rmalization)
conv2_block3_add (Add) (None, 12, 12, 256) 0 ['conv2_block2_out[0][0]',
'conv2_block3_3_bn[0][0]']
conv2_block3_out (Activati (None, 12, 12, 256) 0 ['conv2_block3_add[0][0]']
on)
conv3_block1_1_conv (Conv2 (None, 6, 6, 128) 32896 ['conv2_block3_out[0][0]']
D)
conv3_block1_1_bn (BatchNo (None, 6, 6, 128) 512 ['conv3_block1_1_conv[0][0]']
rmalization)
conv3_block1_1_relu (Activ (None, 6, 6, 128) 0 ['conv3_block1_1_bn[0][0]']
ation)
conv3_block1_2_conv (Conv2 (None, 6, 6, 128) 147584 ['conv3_block1_1_relu[0][0]']
D)
conv3_block1_2_bn (BatchNo (None, 6, 6, 128) 512 ['conv3_block1_2_conv[0][0]']
rmalization)
conv3_block1_2_relu (Activ (None, 6, 6, 128) 0 ['conv3_block1_2_bn[0][0]']
ation)
conv3_block1_0_conv (Conv2 (None, 6, 6, 512) 131584 ['conv2_block3_out[0][0]']
D)
conv3_block1_3_conv (Conv2 (None, 6, 6, 512) 66048 ['conv3_block1_2_relu[0][0]']
D)
conv3_block1_0_bn (BatchNo (None, 6, 6, 512) 2048 ['conv3_block1_0_conv[0][0]']
rmalization)
conv3_block1_3_bn (BatchNo (None, 6, 6, 512) 2048 ['conv3_block1_3_conv[0][0]']
rmalization)
conv3_block1_add (Add) (None, 6, 6, 512) 0 ['conv3_block1_0_bn[0][0]',
'conv3_block1_3_bn[0][0]']
conv3_block1_out (Activati (None, 6, 6, 512) 0 ['conv3_block1_add[0][0]']
on)
conv3_block2_1_conv (Conv2 (None, 6, 6, 128) 65664 ['conv3_block1_out[0][0]']
D)
conv3_block2_1_bn (BatchNo (None, 6, 6, 128) 512 ['conv3_block2_1_conv[0][0]']
rmalization)
conv3_block2_1_relu (Activ (None, 6, 6, 128) 0 ['conv3_block2_1_bn[0][0]']
ation)
conv3_block2_2_conv (Conv2 (None, 6, 6, 128) 147584 ['conv3_block2_1_relu[0][0]']
D)
conv3_block2_2_bn (BatchNo (None, 6, 6, 128) 512 ['conv3_block2_2_conv[0][0]']
rmalization)
conv3_block2_2_relu (Activ (None, 6, 6, 128) 0 ['conv3_block2_2_bn[0][0]']
ation)
conv3_block2_3_conv (Conv2 (None, 6, 6, 512) 66048 ['conv3_block2_2_relu[0][0]']
D)
conv3_block2_3_bn (BatchNo (None, 6, 6, 512) 2048 ['conv3_block2_3_conv[0][0]']
rmalization)
conv3_block2_add (Add) (None, 6, 6, 512) 0 ['conv3_block1_out[0][0]',
'conv3_block2_3_bn[0][0]']
conv3_block2_out (Activati (None, 6, 6, 512) 0 ['conv3_block2_add[0][0]']
on)
conv3_block3_1_conv (Conv2 (None, 6, 6, 128) 65664 ['conv3_block2_out[0][0]']
D)
conv3_block3_1_bn (BatchNo (None, 6, 6, 128) 512 ['conv3_block3_1_conv[0][0]']
rmalization)
conv3_block3_1_relu (Activ (None, 6, 6, 128) 0 ['conv3_block3_1_bn[0][0]']
ation)
conv3_block3_2_conv (Conv2 (None, 6, 6, 128) 147584 ['conv3_block3_1_relu[0][0]']
D)
conv3_block3_2_bn (BatchNo (None, 6, 6, 128) 512 ['conv3_block3_2_conv[0][0]']
rmalization)
conv3_block3_2_relu (Activ (None, 6, 6, 128) 0 ['conv3_block3_2_bn[0][0]']
ation)
conv3_block3_3_conv (Conv2 (None, 6, 6, 512) 66048 ['conv3_block3_2_relu[0][0]']
D)
conv3_block3_3_bn (BatchNo (None, 6, 6, 512) 2048 ['conv3_block3_3_conv[0][0]']
rmalization)
conv3_block3_add (Add) (None, 6, 6, 512) 0 ['conv3_block2_out[0][0]',
'conv3_block3_3_bn[0][0]']
conv3_block3_out (Activati (None, 6, 6, 512) 0 ['conv3_block3_add[0][0]']
on)
conv3_block4_1_conv (Conv2 (None, 6, 6, 128) 65664 ['conv3_block3_out[0][0]']
D)
conv3_block4_1_bn (BatchNo (None, 6, 6, 128) 512 ['conv3_block4_1_conv[0][0]']
rmalization)
conv3_block4_1_relu (Activ (None, 6, 6, 128) 0 ['conv3_block4_1_bn[0][0]']
ation)
conv3_block4_2_conv (Conv2 (None, 6, 6, 128) 147584 ['conv3_block4_1_relu[0][0]']
D)
conv3_block4_2_bn (BatchNo (None, 6, 6, 128) 512 ['conv3_block4_2_conv[0][0]']
rmalization)
conv3_block4_2_relu (Activ (None, 6, 6, 128) 0 ['conv3_block4_2_bn[0][0]']
ation)
conv3_block4_3_conv (Conv2 (None, 6, 6, 512) 66048 ['conv3_block4_2_relu[0][0]']
D)
conv3_block4_3_bn (BatchNo (None, 6, 6, 512) 2048 ['conv3_block4_3_conv[0][0]']
rmalization)
conv3_block4_add (Add) (None, 6, 6, 512) 0 ['conv3_block3_out[0][0]',
'conv3_block4_3_bn[0][0]']
conv3_block4_out (Activati (None, 6, 6, 512) 0 ['conv3_block4_add[0][0]']
on)
conv4_block1_1_conv (Conv2 (None, 3, 3, 256) 131328 ['conv3_block4_out[0][0]']
D)
conv4_block1_1_bn (BatchNo (None, 3, 3, 256) 1024 ['conv4_block1_1_conv[0][0]']
rmalization)
conv4_block1_1_relu (Activ (None, 3, 3, 256) 0 ['conv4_block1_1_bn[0][0]']
ation)
conv4_block1_2_conv (Conv2 (None, 3, 3, 256) 590080 ['conv4_block1_1_relu[0][0]']
D)
conv4_block1_2_bn (BatchNo (None, 3, 3, 256) 1024 ['conv4_block1_2_conv[0][0]']
rmalization)
conv4_block1_2_relu (Activ (None, 3, 3, 256) 0 ['conv4_block1_2_bn[0][0]']
ation)
conv4_block1_0_conv (Conv2 (None, 3, 3, 1024) 525312 ['conv3_block4_out[0][0]']
D)
conv4_block1_3_conv (Conv2 (None, 3, 3, 1024) 263168 ['conv4_block1_2_relu[0][0]']
D)
conv4_block1_0_bn (BatchNo (None, 3, 3, 1024) 4096 ['conv4_block1_0_conv[0][0]']
rmalization)
conv4_block1_3_bn (BatchNo (None, 3, 3, 1024) 4096 ['conv4_block1_3_conv[0][0]']
rmalization)
conv4_block1_add (Add) (None, 3, 3, 1024) 0 ['conv4_block1_0_bn[0][0]',
'conv4_block1_3_bn[0][0]']
conv4_block1_out (Activati (None, 3, 3, 1024) 0 ['conv4_block1_add[0][0]']
on)
conv4_block2_1_conv (Conv2 (None, 3, 3, 256) 262400 ['conv4_block1_out[0][0]']
D)
conv4_block2_1_bn (BatchNo (None, 3, 3, 256) 1024 ['conv4_block2_1_conv[0][0]']
rmalization)
conv4_block2_1_relu (Activ (None, 3, 3, 256) 0 ['conv4_block2_1_bn[0][0]']
ation)
conv4_block2_2_conv (Conv2 (None, 3, 3, 256) 590080 ['conv4_block2_1_relu[0][0]']
D)
conv4_block2_2_bn (BatchNo (None, 3, 3, 256) 1024 ['conv4_block2_2_conv[0][0]']
rmalization)
conv4_block2_2_relu (Activ (None, 3, 3, 256) 0 ['conv4_block2_2_bn[0][0]']
ation)
conv4_block2_3_conv (Conv2 (None, 3, 3, 1024) 263168 ['conv4_block2_2_relu[0][0]']
D)
conv4_block2_3_bn (BatchNo (None, 3, 3, 1024) 4096 ['conv4_block2_3_conv[0][0]']
rmalization)
conv4_block2_add (Add) (None, 3, 3, 1024) 0 ['conv4_block1_out[0][0]',
'conv4_block2_3_bn[0][0]']
conv4_block2_out (Activati (None, 3, 3, 1024) 0 ['conv4_block2_add[0][0]']
on)
conv4_block3_1_conv (Conv2 (None, 3, 3, 256) 262400 ['conv4_block2_out[0][0]']
D)
conv4_block3_1_bn (BatchNo (None, 3, 3, 256) 1024 ['conv4_block3_1_conv[0][0]']
rmalization)
conv4_block3_1_relu (Activ (None, 3, 3, 256) 0 ['conv4_block3_1_bn[0][0]']
ation)
conv4_block3_2_conv (Conv2 (None, 3, 3, 256) 590080 ['conv4_block3_1_relu[0][0]']
D)
conv4_block3_2_bn (BatchNo (None, 3, 3, 256) 1024 ['conv4_block3_2_conv[0][0]']
rmalization)
conv4_block3_2_relu (Activ (None, 3, 3, 256) 0 ['conv4_block3_2_bn[0][0]']
ation)
conv4_block3_3_conv (Conv2 (None, 3, 3, 1024) 263168 ['conv4_block3_2_relu[0][0]']
D)
conv4_block3_3_bn (BatchNo (None, 3, 3, 1024) 4096 ['conv4_block3_3_conv[0][0]']
rmalization)
conv4_block3_add (Add) (None, 3, 3, 1024) 0 ['conv4_block2_out[0][0]',
'conv4_block3_3_bn[0][0]']
conv4_block3_out (Activati (None, 3, 3, 1024) 0 ['conv4_block3_add[0][0]']
on)
conv4_block4_1_conv (Conv2 (None, 3, 3, 256) 262400 ['conv4_block3_out[0][0]']
D)
conv4_block4_1_bn (BatchNo (None, 3, 3, 256) 1024 ['conv4_block4_1_conv[0][0]']
rmalization)
conv4_block4_1_relu (Activ (None, 3, 3, 256) 0 ['conv4_block4_1_bn[0][0]']
ation)
conv4_block4_2_conv (Conv2 (None, 3, 3, 256) 590080 ['conv4_block4_1_relu[0][0]']
D)
conv4_block4_2_bn (BatchNo (None, 3, 3, 256) 1024 ['conv4_block4_2_conv[0][0]']
rmalization)
conv4_block4_2_relu (Activ (None, 3, 3, 256) 0 ['conv4_block4_2_bn[0][0]']
ation)
conv4_block4_3_conv (Conv2 (None, 3, 3, 1024) 263168 ['conv4_block4_2_relu[0][0]']
D)
conv4_block4_3_bn (BatchNo (None, 3, 3, 1024) 4096 ['conv4_block4_3_conv[0][0]']
rmalization)
conv4_block4_add (Add) (None, 3, 3, 1024) 0 ['conv4_block3_out[0][0]',
'conv4_block4_3_bn[0][0]']
conv4_block4_out (Activati (None, 3, 3, 1024) 0 ['conv4_block4_add[0][0]']
on)
conv4_block5_1_conv (Conv2 (None, 3, 3, 256) 262400 ['conv4_block4_out[0][0]']
D)
conv4_block5_1_bn (BatchNo (None, 3, 3, 256) 1024 ['conv4_block5_1_conv[0][0]']
rmalization)
conv4_block5_1_relu (Activ (None, 3, 3, 256) 0 ['conv4_block5_1_bn[0][0]']
ation)
conv4_block5_2_conv (Conv2 (None, 3, 3, 256) 590080 ['conv4_block5_1_relu[0][0]']
D)
conv4_block5_2_bn (BatchNo (None, 3, 3, 256) 1024 ['conv4_block5_2_conv[0][0]']
rmalization)
conv4_block5_2_relu (Activ (None, 3, 3, 256) 0 ['conv4_block5_2_bn[0][0]']
ation)
conv4_block5_3_conv (Conv2 (None, 3, 3, 1024) 263168 ['conv4_block5_2_relu[0][0]']
D)
conv4_block5_3_bn (BatchNo (None, 3, 3, 1024) 4096 ['conv4_block5_3_conv[0][0]']
rmalization)
conv4_block5_add (Add) (None, 3, 3, 1024) 0 ['conv4_block4_out[0][0]',
'conv4_block5_3_bn[0][0]']
conv4_block5_out (Activati (None, 3, 3, 1024) 0 ['conv4_block5_add[0][0]']
on)
conv4_block6_1_conv (Conv2 (None, 3, 3, 256) 262400 ['conv4_block5_out[0][0]']
D)
conv4_block6_1_bn (BatchNo (None, 3, 3, 256) 1024 ['conv4_block6_1_conv[0][0]']
rmalization)
conv4_block6_1_relu (Activ (None, 3, 3, 256) 0 ['conv4_block6_1_bn[0][0]']
ation)
conv4_block6_2_conv (Conv2 (None, 3, 3, 256) 590080 ['conv4_block6_1_relu[0][0]']
D)
conv4_block6_2_bn (BatchNo (None, 3, 3, 256) 1024 ['conv4_block6_2_conv[0][0]']
rmalization)
conv4_block6_2_relu (Activ (None, 3, 3, 256) 0 ['conv4_block6_2_bn[0][0]']
ation)
conv4_block6_3_conv (Conv2 (None, 3, 3, 1024) 263168 ['conv4_block6_2_relu[0][0]']
D)
conv4_block6_3_bn (BatchNo (None, 3, 3, 1024) 4096 ['conv4_block6_3_conv[0][0]']
rmalization)
conv4_block6_add (Add) (None, 3, 3, 1024) 0 ['conv4_block5_out[0][0]',
'conv4_block6_3_bn[0][0]']
conv4_block6_out (Activati (None, 3, 3, 1024) 0 ['conv4_block6_add[0][0]']
on)
conv5_block1_1_conv (Conv2 (None, 2, 2, 512) 524800 ['conv4_block6_out[0][0]']
D)
conv5_block1_1_bn (BatchNo (None, 2, 2, 512) 2048 ['conv5_block1_1_conv[0][0]']
rmalization)
conv5_block1_1_relu (Activ (None, 2, 2, 512) 0 ['conv5_block1_1_bn[0][0]']
ation)
conv5_block1_2_conv (Conv2 (None, 2, 2, 512) 2359808 ['conv5_block1_1_relu[0][0]']
D)
conv5_block1_2_bn (BatchNo (None, 2, 2, 512) 2048 ['conv5_block1_2_conv[0][0]']
rmalization)
conv5_block1_2_relu (Activ (None, 2, 2, 512) 0 ['conv5_block1_2_bn[0][0]']
ation)
conv5_block1_0_conv (Conv2 (None, 2, 2, 2048) 2099200 ['conv4_block6_out[0][0]']
D)
conv5_block1_3_conv (Conv2 (None, 2, 2, 2048) 1050624 ['conv5_block1_2_relu[0][0]']
D)
conv5_block1_0_bn (BatchNo (None, 2, 2, 2048) 8192 ['conv5_block1_0_conv[0][0]']
rmalization)
conv5_block1_3_bn (BatchNo (None, 2, 2, 2048) 8192 ['conv5_block1_3_conv[0][0]']
rmalization)
conv5_block1_add (Add) (None, 2, 2, 2048) 0 ['conv5_block1_0_bn[0][0]',
'conv5_block1_3_bn[0][0]']
conv5_block1_out (Activati (None, 2, 2, 2048) 0 ['conv5_block1_add[0][0]']
on)
conv5_block2_1_conv (Conv2 (None, 2, 2, 512) 1049088 ['conv5_block1_out[0][0]']
D)
conv5_block2_1_bn (BatchNo (None, 2, 2, 512) 2048 ['conv5_block2_1_conv[0][0]']
rmalization)
conv5_block2_1_relu (Activ (None, 2, 2, 512) 0 ['conv5_block2_1_bn[0][0]']
ation)
conv5_block2_2_conv (Conv2 (None, 2, 2, 512) 2359808 ['conv5_block2_1_relu[0][0]']
D)
conv5_block2_2_bn (BatchNo (None, 2, 2, 512) 2048 ['conv5_block2_2_conv[0][0]']
rmalization)
conv5_block2_2_relu (Activ (None, 2, 2, 512) 0 ['conv5_block2_2_bn[0][0]']
ation)
conv5_block2_3_conv (Conv2 (None, 2, 2, 2048) 1050624 ['conv5_block2_2_relu[0][0]']
D)
conv5_block2_3_bn (BatchNo (None, 2, 2, 2048) 8192 ['conv5_block2_3_conv[0][0]']
rmalization)
conv5_block2_add (Add) (None, 2, 2, 2048) 0 ['conv5_block1_out[0][0]',
'conv5_block2_3_bn[0][0]']
conv5_block2_out (Activati (None, 2, 2, 2048) 0 ['conv5_block2_add[0][0]']
on)
conv5_block3_1_conv (Conv2 (None, 2, 2, 512) 1049088 ['conv5_block2_out[0][0]']
D)
conv5_block3_1_bn (BatchNo (None, 2, 2, 512) 2048 ['conv5_block3_1_conv[0][0]']
rmalization)
conv5_block3_1_relu (Activ (None, 2, 2, 512) 0 ['conv5_block3_1_bn[0][0]']
ation)
conv5_block3_2_conv (Conv2 (None, 2, 2, 512) 2359808 ['conv5_block3_1_relu[0][0]']
D)
conv5_block3_2_bn (BatchNo (None, 2, 2, 512) 2048 ['conv5_block3_2_conv[0][0]']
rmalization)
conv5_block3_2_relu (Activ (None, 2, 2, 512) 0 ['conv5_block3_2_bn[0][0]']
ation)
conv5_block3_3_conv (Conv2 (None, 2, 2, 2048) 1050624 ['conv5_block3_2_relu[0][0]']
D)
conv5_block3_3_bn (BatchNo (None, 2, 2, 2048) 8192 ['conv5_block3_3_conv[0][0]']
rmalization)
conv5_block3_add (Add) (None, 2, 2, 2048) 0 ['conv5_block2_out[0][0]',
'conv5_block3_3_bn[0][0]']
conv5_block3_out (Activati (None, 2, 2, 2048) 0 ['conv5_block3_add[0][0]']
on)
==================================================================================================
Total params: 23587712 (89.98 MB)
Trainable params: 23534592 (89.78 MB)
Non-trainable params: 53120 (207.50 KB)
__________________________________________________________________________________________________
# Visualize the model as blocks
plot_model(Resnet, to_file='model.png', show_shapes=True, show_layer_names=True)
# Set ResNet layers to non-trainable
Resnet.trainable = False
# Rebuild the model
x = Resnet.output
# Add Global Average Pooling layer
x = GlobalAveragePooling2D()(x)
# Flatten the output
x = Flatten()(x)
# Add a Dense layer with 256 neurons
x = Dense(256, activation='relu')(x)
# Add a Dense Layer with 128 neurons
x = Dense(128, activation='relu')(x)
# Add a DropOut layer with a drop out ratio of 0.3
x = Dropout(0.3)(x)
# Add a Dense Layer with 64 neurons
x = Dense(64, activation='relu')(x)
# Add a Batch Normalization layer
x = BatchNormalization()(x)
# Add the final dense layer with 4 neurons and 'softmax' activation for multi-class classification
pred = Dense(4, activation='softmax')(x)
# Initializing the model
resnetmodel = Model(inputs=Resnet.input, outputs=pred)
# ModelCheckpoint is used to save the model at certain intervals
checkpoint_resnet = ModelCheckpoint("resnet.h5", # file path where the model will be saved
monitor='val_accuracy', # means the callback monitors the validation accuracy
verbose=1, # model is saved only when the metric (val_acc) has improved
save_best_only=True,
mode='max')
# Configured callbacks will be passed to the model during training
callbacks_list_resnet = [early_stopping, checkpoint_resnet, reduce_learningrate]
# Number of iterations over the entire dataset that the training process should run
epochs = 20
# Compiling the CNN model with categorical crossentropy as loss function, Adam Optimizer
# with 0.001 learning rate, and set metrics set to 'accuracy'.
resnetmodel.compile(loss='categorical_crossentropy',
optimizer=legacy.Adam(learning_rate = 0.001),
metrics=['accuracy'])
# Fitting the model with required parameters above, saving model into history_resnet variable
history_resnet = resnetmodel.fit(
train_set_pre,
steps_per_epoch = train_set_pre.samples // train_set_pre.batch_size,
validation_data = validation_set_pre,
validation_steps = validation_set_pre.samples // validation_set_pre.batch_size,
epochs = 20,
verbose = 1,
callbacks = callbacks_list_resnet
)
Epoch 1/20 472/472 [==============================] - ETA: 0s - loss: 1.4481 - accuracy: 0.2605 Epoch 1: val_accuracy improved from -inf to 0.36996, saving model to resnet.h5 472/472 [==============================] - 47s 96ms/step - loss: 1.4481 - accuracy: 0.2605 - val_loss: 1.3482 - val_accuracy: 0.3700 - lr: 0.0010 Epoch 2/20 472/472 [==============================] - ETA: 0s - loss: 1.3908 - accuracy: 0.2780 Epoch 2: val_accuracy did not improve from 0.36996 472/472 [==============================] - 47s 99ms/step - loss: 1.3908 - accuracy: 0.2780 - val_loss: 1.3547 - val_accuracy: 0.2639 - lr: 0.0010 Epoch 3/20 472/472 [==============================] - ETA: 0s - loss: 1.3560 - accuracy: 0.3034 Epoch 3: val_accuracy did not improve from 0.36996 472/472 [==============================] - 51s 107ms/step - loss: 1.3560 - accuracy: 0.3034 - val_loss: 1.3632 - val_accuracy: 0.2653 - lr: 0.0010 Epoch 4/20 472/472 [==============================] - ETA: 0s - loss: 1.3313 - accuracy: 0.3308 Epoch 4: val_accuracy did not improve from 0.36996 472/472 [==============================] - 45s 95ms/step - loss: 1.3313 - accuracy: 0.3308 - val_loss: 1.3288 - val_accuracy: 0.3010 - lr: 0.0010 Epoch 5/20 472/472 [==============================] - ETA: 0s - loss: 1.3136 - accuracy: 0.3434 Epoch 5: val_accuracy did not improve from 0.36996 472/472 [==============================] - 46s 97ms/step - loss: 1.3136 - accuracy: 0.3434 - val_loss: 1.3745 - val_accuracy: 0.2597 - lr: 0.0010 Epoch 6/20 472/472 [==============================] - ETA: 0s - loss: 1.2930 - accuracy: 0.3601 Epoch 6: val_accuracy did not improve from 0.36996 472/472 [==============================] - 45s 94ms/step - loss: 1.2930 - accuracy: 0.3601 - val_loss: 1.3816 - val_accuracy: 0.2909 - lr: 0.0010 Epoch 7/20 472/472 [==============================] - ETA: 0s - loss: 1.2777 - accuracy: 0.3820Restoring model weights from the end of the best epoch: 4. Epoch 7: val_accuracy did not improve from 0.36996 Epoch 7: ReduceLROnPlateau reducing learning rate to 0.00020000000949949026. 472/472 [==============================] - 46s 97ms/step - loss: 1.2777 - accuracy: 0.3820 - val_loss: 1.3505 - val_accuracy: 0.3075 - lr: 0.0010 Epoch 7: early stopping
# Load the saved model if notebook session gets disconnected
resnetmodel = load_model("resnet.h5")
# Evaluate the model on the test data
test_loss, test_accuracy = resnetmodel.evaluate(test_set_pre, verbose=1)
# Print the results
print("Test Loss:", test_loss)
print("Test Accuracy:", test_accuracy)
4/4 [==============================] - 0s 80ms/step - loss: 1.3008 - accuracy: 0.3750 Test Loss: 1.300801396369934 Test Accuracy: 0.375
# Plotting the Training and Validation accuracies of ResNet V2
plt.plot(history_resnet.history['accuracy'])
plt.plot(history_resnet.history['val_accuracy'])
plt.title('ResNet V2 Transfer Learning Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc = 'upper left')
# Display the plot
plt.show()
Observations and Insights:
# Clearing backend
backend.clear_session()
# Reset seed for number generators using reset_seed() function
reset_seed(fixed_seed)
Random number generators seed has been set to 42
# Importing EfficientNet Model libraries
import tensorflow as tf
import tensorflow.keras.applications as ap
from tensorflow.keras import Model
# Variable 'EfficientNet' will contain the EfficientNet model
EfficientNet = ap.EfficientNetV2B2(include_top=False,
weights="imagenet",
input_shape= (img_size[0], img_size[1], 3))
# Print imported model's summary
EfficientNet.summary()
Model: "efficientnetv2-b2"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_1 (InputLayer) [(None, 48, 48, 3)] 0 []
rescaling (Rescaling) (None, 48, 48, 3) 0 ['input_1[0][0]']
normalization (Normalizati (None, 48, 48, 3) 0 ['rescaling[0][0]']
on)
stem_conv (Conv2D) (None, 24, 24, 32) 864 ['normalization[0][0]']
stem_bn (BatchNormalizatio (None, 24, 24, 32) 128 ['stem_conv[0][0]']
n)
stem_activation (Activatio (None, 24, 24, 32) 0 ['stem_bn[0][0]']
n)
block1a_project_conv (Conv (None, 24, 24, 16) 4608 ['stem_activation[0][0]']
2D)
block1a_project_bn (BatchN (None, 24, 24, 16) 64 ['block1a_project_conv[0][0]']
ormalization)
block1a_project_activation (None, 24, 24, 16) 0 ['block1a_project_bn[0][0]']
(Activation)
block1b_project_conv (Conv (None, 24, 24, 16) 2304 ['block1a_project_activation[0
2D) ][0]']
block1b_project_bn (BatchN (None, 24, 24, 16) 64 ['block1b_project_conv[0][0]']
ormalization)
block1b_project_activation (None, 24, 24, 16) 0 ['block1b_project_bn[0][0]']
(Activation)
block1b_drop (Dropout) (None, 24, 24, 16) 0 ['block1b_project_activation[0
][0]']
block1b_add (Add) (None, 24, 24, 16) 0 ['block1b_drop[0][0]',
'block1a_project_activation[0
][0]']
block2a_expand_conv (Conv2 (None, 12, 12, 64) 9216 ['block1b_add[0][0]']
D)
block2a_expand_bn (BatchNo (None, 12, 12, 64) 256 ['block2a_expand_conv[0][0]']
rmalization)
block2a_expand_activation (None, 12, 12, 64) 0 ['block2a_expand_bn[0][0]']
(Activation)
block2a_project_conv (Conv (None, 12, 12, 32) 2048 ['block2a_expand_activation[0]
2D) [0]']
block2a_project_bn (BatchN (None, 12, 12, 32) 128 ['block2a_project_conv[0][0]']
ormalization)
block2b_expand_conv (Conv2 (None, 12, 12, 128) 36864 ['block2a_project_bn[0][0]']
D)
block2b_expand_bn (BatchNo (None, 12, 12, 128) 512 ['block2b_expand_conv[0][0]']
rmalization)
block2b_expand_activation (None, 12, 12, 128) 0 ['block2b_expand_bn[0][0]']
(Activation)
block2b_project_conv (Conv (None, 12, 12, 32) 4096 ['block2b_expand_activation[0]
2D) [0]']
block2b_project_bn (BatchN (None, 12, 12, 32) 128 ['block2b_project_conv[0][0]']
ormalization)
block2b_drop (Dropout) (None, 12, 12, 32) 0 ['block2b_project_bn[0][0]']
block2b_add (Add) (None, 12, 12, 32) 0 ['block2b_drop[0][0]',
'block2a_project_bn[0][0]']
block2c_expand_conv (Conv2 (None, 12, 12, 128) 36864 ['block2b_add[0][0]']
D)
block2c_expand_bn (BatchNo (None, 12, 12, 128) 512 ['block2c_expand_conv[0][0]']
rmalization)
block2c_expand_activation (None, 12, 12, 128) 0 ['block2c_expand_bn[0][0]']
(Activation)
block2c_project_conv (Conv (None, 12, 12, 32) 4096 ['block2c_expand_activation[0]
2D) [0]']
block2c_project_bn (BatchN (None, 12, 12, 32) 128 ['block2c_project_conv[0][0]']
ormalization)
block2c_drop (Dropout) (None, 12, 12, 32) 0 ['block2c_project_bn[0][0]']
block2c_add (Add) (None, 12, 12, 32) 0 ['block2c_drop[0][0]',
'block2b_add[0][0]']
block3a_expand_conv (Conv2 (None, 6, 6, 128) 36864 ['block2c_add[0][0]']
D)
block3a_expand_bn (BatchNo (None, 6, 6, 128) 512 ['block3a_expand_conv[0][0]']
rmalization)
block3a_expand_activation (None, 6, 6, 128) 0 ['block3a_expand_bn[0][0]']
(Activation)
block3a_project_conv (Conv (None, 6, 6, 56) 7168 ['block3a_expand_activation[0]
2D) [0]']
block3a_project_bn (BatchN (None, 6, 6, 56) 224 ['block3a_project_conv[0][0]']
ormalization)
block3b_expand_conv (Conv2 (None, 6, 6, 224) 112896 ['block3a_project_bn[0][0]']
D)
block3b_expand_bn (BatchNo (None, 6, 6, 224) 896 ['block3b_expand_conv[0][0]']
rmalization)
block3b_expand_activation (None, 6, 6, 224) 0 ['block3b_expand_bn[0][0]']
(Activation)
block3b_project_conv (Conv (None, 6, 6, 56) 12544 ['block3b_expand_activation[0]
2D) [0]']
block3b_project_bn (BatchN (None, 6, 6, 56) 224 ['block3b_project_conv[0][0]']
ormalization)
block3b_drop (Dropout) (None, 6, 6, 56) 0 ['block3b_project_bn[0][0]']
block3b_add (Add) (None, 6, 6, 56) 0 ['block3b_drop[0][0]',
'block3a_project_bn[0][0]']
block3c_expand_conv (Conv2 (None, 6, 6, 224) 112896 ['block3b_add[0][0]']
D)
block3c_expand_bn (BatchNo (None, 6, 6, 224) 896 ['block3c_expand_conv[0][0]']
rmalization)
block3c_expand_activation (None, 6, 6, 224) 0 ['block3c_expand_bn[0][0]']
(Activation)
block3c_project_conv (Conv (None, 6, 6, 56) 12544 ['block3c_expand_activation[0]
2D) [0]']
block3c_project_bn (BatchN (None, 6, 6, 56) 224 ['block3c_project_conv[0][0]']
ormalization)
block3c_drop (Dropout) (None, 6, 6, 56) 0 ['block3c_project_bn[0][0]']
block3c_add (Add) (None, 6, 6, 56) 0 ['block3c_drop[0][0]',
'block3b_add[0][0]']
block4a_expand_conv (Conv2 (None, 6, 6, 224) 12544 ['block3c_add[0][0]']
D)
block4a_expand_bn (BatchNo (None, 6, 6, 224) 896 ['block4a_expand_conv[0][0]']
rmalization)
block4a_expand_activation (None, 6, 6, 224) 0 ['block4a_expand_bn[0][0]']
(Activation)
block4a_dwconv2 (Depthwise (None, 3, 3, 224) 2016 ['block4a_expand_activation[0]
Conv2D) [0]']
block4a_bn (BatchNormaliza (None, 3, 3, 224) 896 ['block4a_dwconv2[0][0]']
tion)
block4a_activation (Activa (None, 3, 3, 224) 0 ['block4a_bn[0][0]']
tion)
block4a_se_squeeze (Global (None, 224) 0 ['block4a_activation[0][0]']
AveragePooling2D)
block4a_se_reshape (Reshap (None, 1, 1, 224) 0 ['block4a_se_squeeze[0][0]']
e)
block4a_se_reduce (Conv2D) (None, 1, 1, 14) 3150 ['block4a_se_reshape[0][0]']
block4a_se_expand (Conv2D) (None, 1, 1, 224) 3360 ['block4a_se_reduce[0][0]']
block4a_se_excite (Multipl (None, 3, 3, 224) 0 ['block4a_activation[0][0]',
y) 'block4a_se_expand[0][0]']
block4a_project_conv (Conv (None, 3, 3, 104) 23296 ['block4a_se_excite[0][0]']
2D)
block4a_project_bn (BatchN (None, 3, 3, 104) 416 ['block4a_project_conv[0][0]']
ormalization)
block4b_expand_conv (Conv2 (None, 3, 3, 416) 43264 ['block4a_project_bn[0][0]']
D)
block4b_expand_bn (BatchNo (None, 3, 3, 416) 1664 ['block4b_expand_conv[0][0]']
rmalization)
block4b_expand_activation (None, 3, 3, 416) 0 ['block4b_expand_bn[0][0]']
(Activation)
block4b_dwconv2 (Depthwise (None, 3, 3, 416) 3744 ['block4b_expand_activation[0]
Conv2D) [0]']
block4b_bn (BatchNormaliza (None, 3, 3, 416) 1664 ['block4b_dwconv2[0][0]']
tion)
block4b_activation (Activa (None, 3, 3, 416) 0 ['block4b_bn[0][0]']
tion)
block4b_se_squeeze (Global (None, 416) 0 ['block4b_activation[0][0]']
AveragePooling2D)
block4b_se_reshape (Reshap (None, 1, 1, 416) 0 ['block4b_se_squeeze[0][0]']
e)
block4b_se_reduce (Conv2D) (None, 1, 1, 26) 10842 ['block4b_se_reshape[0][0]']
block4b_se_expand (Conv2D) (None, 1, 1, 416) 11232 ['block4b_se_reduce[0][0]']
block4b_se_excite (Multipl (None, 3, 3, 416) 0 ['block4b_activation[0][0]',
y) 'block4b_se_expand[0][0]']
block4b_project_conv (Conv (None, 3, 3, 104) 43264 ['block4b_se_excite[0][0]']
2D)
block4b_project_bn (BatchN (None, 3, 3, 104) 416 ['block4b_project_conv[0][0]']
ormalization)
block4b_drop (Dropout) (None, 3, 3, 104) 0 ['block4b_project_bn[0][0]']
block4b_add (Add) (None, 3, 3, 104) 0 ['block4b_drop[0][0]',
'block4a_project_bn[0][0]']
block4c_expand_conv (Conv2 (None, 3, 3, 416) 43264 ['block4b_add[0][0]']
D)
block4c_expand_bn (BatchNo (None, 3, 3, 416) 1664 ['block4c_expand_conv[0][0]']
rmalization)
block4c_expand_activation (None, 3, 3, 416) 0 ['block4c_expand_bn[0][0]']
(Activation)
block4c_dwconv2 (Depthwise (None, 3, 3, 416) 3744 ['block4c_expand_activation[0]
Conv2D) [0]']
block4c_bn (BatchNormaliza (None, 3, 3, 416) 1664 ['block4c_dwconv2[0][0]']
tion)
block4c_activation (Activa (None, 3, 3, 416) 0 ['block4c_bn[0][0]']
tion)
block4c_se_squeeze (Global (None, 416) 0 ['block4c_activation[0][0]']
AveragePooling2D)
block4c_se_reshape (Reshap (None, 1, 1, 416) 0 ['block4c_se_squeeze[0][0]']
e)
block4c_se_reduce (Conv2D) (None, 1, 1, 26) 10842 ['block4c_se_reshape[0][0]']
block4c_se_expand (Conv2D) (None, 1, 1, 416) 11232 ['block4c_se_reduce[0][0]']
block4c_se_excite (Multipl (None, 3, 3, 416) 0 ['block4c_activation[0][0]',
y) 'block4c_se_expand[0][0]']
block4c_project_conv (Conv (None, 3, 3, 104) 43264 ['block4c_se_excite[0][0]']
2D)
block4c_project_bn (BatchN (None, 3, 3, 104) 416 ['block4c_project_conv[0][0]']
ormalization)
block4c_drop (Dropout) (None, 3, 3, 104) 0 ['block4c_project_bn[0][0]']
block4c_add (Add) (None, 3, 3, 104) 0 ['block4c_drop[0][0]',
'block4b_add[0][0]']
block4d_expand_conv (Conv2 (None, 3, 3, 416) 43264 ['block4c_add[0][0]']
D)
block4d_expand_bn (BatchNo (None, 3, 3, 416) 1664 ['block4d_expand_conv[0][0]']
rmalization)
block4d_expand_activation (None, 3, 3, 416) 0 ['block4d_expand_bn[0][0]']
(Activation)
block4d_dwconv2 (Depthwise (None, 3, 3, 416) 3744 ['block4d_expand_activation[0]
Conv2D) [0]']
block4d_bn (BatchNormaliza (None, 3, 3, 416) 1664 ['block4d_dwconv2[0][0]']
tion)
block4d_activation (Activa (None, 3, 3, 416) 0 ['block4d_bn[0][0]']
tion)
block4d_se_squeeze (Global (None, 416) 0 ['block4d_activation[0][0]']
AveragePooling2D)
block4d_se_reshape (Reshap (None, 1, 1, 416) 0 ['block4d_se_squeeze[0][0]']
e)
block4d_se_reduce (Conv2D) (None, 1, 1, 26) 10842 ['block4d_se_reshape[0][0]']
block4d_se_expand (Conv2D) (None, 1, 1, 416) 11232 ['block4d_se_reduce[0][0]']
block4d_se_excite (Multipl (None, 3, 3, 416) 0 ['block4d_activation[0][0]',
y) 'block4d_se_expand[0][0]']
block4d_project_conv (Conv (None, 3, 3, 104) 43264 ['block4d_se_excite[0][0]']
2D)
block4d_project_bn (BatchN (None, 3, 3, 104) 416 ['block4d_project_conv[0][0]']
ormalization)
block4d_drop (Dropout) (None, 3, 3, 104) 0 ['block4d_project_bn[0][0]']
block4d_add (Add) (None, 3, 3, 104) 0 ['block4d_drop[0][0]',
'block4c_add[0][0]']
block5a_expand_conv (Conv2 (None, 3, 3, 624) 64896 ['block4d_add[0][0]']
D)
block5a_expand_bn (BatchNo (None, 3, 3, 624) 2496 ['block5a_expand_conv[0][0]']
rmalization)
block5a_expand_activation (None, 3, 3, 624) 0 ['block5a_expand_bn[0][0]']
(Activation)
block5a_dwconv2 (Depthwise (None, 3, 3, 624) 5616 ['block5a_expand_activation[0]
Conv2D) [0]']
block5a_bn (BatchNormaliza (None, 3, 3, 624) 2496 ['block5a_dwconv2[0][0]']
tion)
block5a_activation (Activa (None, 3, 3, 624) 0 ['block5a_bn[0][0]']
tion)
block5a_se_squeeze (Global (None, 624) 0 ['block5a_activation[0][0]']
AveragePooling2D)
block5a_se_reshape (Reshap (None, 1, 1, 624) 0 ['block5a_se_squeeze[0][0]']
e)
block5a_se_reduce (Conv2D) (None, 1, 1, 26) 16250 ['block5a_se_reshape[0][0]']
block5a_se_expand (Conv2D) (None, 1, 1, 624) 16848 ['block5a_se_reduce[0][0]']
block5a_se_excite (Multipl (None, 3, 3, 624) 0 ['block5a_activation[0][0]',
y) 'block5a_se_expand[0][0]']
block5a_project_conv (Conv (None, 3, 3, 120) 74880 ['block5a_se_excite[0][0]']
2D)
block5a_project_bn (BatchN (None, 3, 3, 120) 480 ['block5a_project_conv[0][0]']
ormalization)
block5b_expand_conv (Conv2 (None, 3, 3, 720) 86400 ['block5a_project_bn[0][0]']
D)
block5b_expand_bn (BatchNo (None, 3, 3, 720) 2880 ['block5b_expand_conv[0][0]']
rmalization)
block5b_expand_activation (None, 3, 3, 720) 0 ['block5b_expand_bn[0][0]']
(Activation)
block5b_dwconv2 (Depthwise (None, 3, 3, 720) 6480 ['block5b_expand_activation[0]
Conv2D) [0]']
block5b_bn (BatchNormaliza (None, 3, 3, 720) 2880 ['block5b_dwconv2[0][0]']
tion)
block5b_activation (Activa (None, 3, 3, 720) 0 ['block5b_bn[0][0]']
tion)
block5b_se_squeeze (Global (None, 720) 0 ['block5b_activation[0][0]']
AveragePooling2D)
block5b_se_reshape (Reshap (None, 1, 1, 720) 0 ['block5b_se_squeeze[0][0]']
e)
block5b_se_reduce (Conv2D) (None, 1, 1, 30) 21630 ['block5b_se_reshape[0][0]']
block5b_se_expand (Conv2D) (None, 1, 1, 720) 22320 ['block5b_se_reduce[0][0]']
block5b_se_excite (Multipl (None, 3, 3, 720) 0 ['block5b_activation[0][0]',
y) 'block5b_se_expand[0][0]']
block5b_project_conv (Conv (None, 3, 3, 120) 86400 ['block5b_se_excite[0][0]']
2D)
block5b_project_bn (BatchN (None, 3, 3, 120) 480 ['block5b_project_conv[0][0]']
ormalization)
block5b_drop (Dropout) (None, 3, 3, 120) 0 ['block5b_project_bn[0][0]']
block5b_add (Add) (None, 3, 3, 120) 0 ['block5b_drop[0][0]',
'block5a_project_bn[0][0]']
block5c_expand_conv (Conv2 (None, 3, 3, 720) 86400 ['block5b_add[0][0]']
D)
block5c_expand_bn (BatchNo (None, 3, 3, 720) 2880 ['block5c_expand_conv[0][0]']
rmalization)
block5c_expand_activation (None, 3, 3, 720) 0 ['block5c_expand_bn[0][0]']
(Activation)
block5c_dwconv2 (Depthwise (None, 3, 3, 720) 6480 ['block5c_expand_activation[0]
Conv2D) [0]']
block5c_bn (BatchNormaliza (None, 3, 3, 720) 2880 ['block5c_dwconv2[0][0]']
tion)
block5c_activation (Activa (None, 3, 3, 720) 0 ['block5c_bn[0][0]']
tion)
block5c_se_squeeze (Global (None, 720) 0 ['block5c_activation[0][0]']
AveragePooling2D)
block5c_se_reshape (Reshap (None, 1, 1, 720) 0 ['block5c_se_squeeze[0][0]']
e)
block5c_se_reduce (Conv2D) (None, 1, 1, 30) 21630 ['block5c_se_reshape[0][0]']
block5c_se_expand (Conv2D) (None, 1, 1, 720) 22320 ['block5c_se_reduce[0][0]']
block5c_se_excite (Multipl (None, 3, 3, 720) 0 ['block5c_activation[0][0]',
y) 'block5c_se_expand[0][0]']
block5c_project_conv (Conv (None, 3, 3, 120) 86400 ['block5c_se_excite[0][0]']
2D)
block5c_project_bn (BatchN (None, 3, 3, 120) 480 ['block5c_project_conv[0][0]']
ormalization)
block5c_drop (Dropout) (None, 3, 3, 120) 0 ['block5c_project_bn[0][0]']
block5c_add (Add) (None, 3, 3, 120) 0 ['block5c_drop[0][0]',
'block5b_add[0][0]']
block5d_expand_conv (Conv2 (None, 3, 3, 720) 86400 ['block5c_add[0][0]']
D)
block5d_expand_bn (BatchNo (None, 3, 3, 720) 2880 ['block5d_expand_conv[0][0]']
rmalization)
block5d_expand_activation (None, 3, 3, 720) 0 ['block5d_expand_bn[0][0]']
(Activation)
block5d_dwconv2 (Depthwise (None, 3, 3, 720) 6480 ['block5d_expand_activation[0]
Conv2D) [0]']
block5d_bn (BatchNormaliza (None, 3, 3, 720) 2880 ['block5d_dwconv2[0][0]']
tion)
block5d_activation (Activa (None, 3, 3, 720) 0 ['block5d_bn[0][0]']
tion)
block5d_se_squeeze (Global (None, 720) 0 ['block5d_activation[0][0]']
AveragePooling2D)
block5d_se_reshape (Reshap (None, 1, 1, 720) 0 ['block5d_se_squeeze[0][0]']
e)
block5d_se_reduce (Conv2D) (None, 1, 1, 30) 21630 ['block5d_se_reshape[0][0]']
block5d_se_expand (Conv2D) (None, 1, 1, 720) 22320 ['block5d_se_reduce[0][0]']
block5d_se_excite (Multipl (None, 3, 3, 720) 0 ['block5d_activation[0][0]',
y) 'block5d_se_expand[0][0]']
block5d_project_conv (Conv (None, 3, 3, 120) 86400 ['block5d_se_excite[0][0]']
2D)
block5d_project_bn (BatchN (None, 3, 3, 120) 480 ['block5d_project_conv[0][0]']
ormalization)
block5d_drop (Dropout) (None, 3, 3, 120) 0 ['block5d_project_bn[0][0]']
block5d_add (Add) (None, 3, 3, 120) 0 ['block5d_drop[0][0]',
'block5c_add[0][0]']
block5e_expand_conv (Conv2 (None, 3, 3, 720) 86400 ['block5d_add[0][0]']
D)
block5e_expand_bn (BatchNo (None, 3, 3, 720) 2880 ['block5e_expand_conv[0][0]']
rmalization)
block5e_expand_activation (None, 3, 3, 720) 0 ['block5e_expand_bn[0][0]']
(Activation)
block5e_dwconv2 (Depthwise (None, 3, 3, 720) 6480 ['block5e_expand_activation[0]
Conv2D) [0]']
block5e_bn (BatchNormaliza (None, 3, 3, 720) 2880 ['block5e_dwconv2[0][0]']
tion)
block5e_activation (Activa (None, 3, 3, 720) 0 ['block5e_bn[0][0]']
tion)
block5e_se_squeeze (Global (None, 720) 0 ['block5e_activation[0][0]']
AveragePooling2D)
block5e_se_reshape (Reshap (None, 1, 1, 720) 0 ['block5e_se_squeeze[0][0]']
e)
block5e_se_reduce (Conv2D) (None, 1, 1, 30) 21630 ['block5e_se_reshape[0][0]']
block5e_se_expand (Conv2D) (None, 1, 1, 720) 22320 ['block5e_se_reduce[0][0]']
block5e_se_excite (Multipl (None, 3, 3, 720) 0 ['block5e_activation[0][0]',
y) 'block5e_se_expand[0][0]']
block5e_project_conv (Conv (None, 3, 3, 120) 86400 ['block5e_se_excite[0][0]']
2D)
block5e_project_bn (BatchN (None, 3, 3, 120) 480 ['block5e_project_conv[0][0]']
ormalization)
block5e_drop (Dropout) (None, 3, 3, 120) 0 ['block5e_project_bn[0][0]']
block5e_add (Add) (None, 3, 3, 120) 0 ['block5e_drop[0][0]',
'block5d_add[0][0]']
block5f_expand_conv (Conv2 (None, 3, 3, 720) 86400 ['block5e_add[0][0]']
D)
block5f_expand_bn (BatchNo (None, 3, 3, 720) 2880 ['block5f_expand_conv[0][0]']
rmalization)
block5f_expand_activation (None, 3, 3, 720) 0 ['block5f_expand_bn[0][0]']
(Activation)
block5f_dwconv2 (Depthwise (None, 3, 3, 720) 6480 ['block5f_expand_activation[0]
Conv2D) [0]']
block5f_bn (BatchNormaliza (None, 3, 3, 720) 2880 ['block5f_dwconv2[0][0]']
tion)
block5f_activation (Activa (None, 3, 3, 720) 0 ['block5f_bn[0][0]']
tion)
block5f_se_squeeze (Global (None, 720) 0 ['block5f_activation[0][0]']
AveragePooling2D)
block5f_se_reshape (Reshap (None, 1, 1, 720) 0 ['block5f_se_squeeze[0][0]']
e)
block5f_se_reduce (Conv2D) (None, 1, 1, 30) 21630 ['block5f_se_reshape[0][0]']
block5f_se_expand (Conv2D) (None, 1, 1, 720) 22320 ['block5f_se_reduce[0][0]']
block5f_se_excite (Multipl (None, 3, 3, 720) 0 ['block5f_activation[0][0]',
y) 'block5f_se_expand[0][0]']
block5f_project_conv (Conv (None, 3, 3, 120) 86400 ['block5f_se_excite[0][0]']
2D)
block5f_project_bn (BatchN (None, 3, 3, 120) 480 ['block5f_project_conv[0][0]']
ormalization)
block5f_drop (Dropout) (None, 3, 3, 120) 0 ['block5f_project_bn[0][0]']
block5f_add (Add) (None, 3, 3, 120) 0 ['block5f_drop[0][0]',
'block5e_add[0][0]']
block6a_expand_conv (Conv2 (None, 3, 3, 720) 86400 ['block5f_add[0][0]']
D)
block6a_expand_bn (BatchNo (None, 3, 3, 720) 2880 ['block6a_expand_conv[0][0]']
rmalization)
block6a_expand_activation (None, 3, 3, 720) 0 ['block6a_expand_bn[0][0]']
(Activation)
block6a_dwconv2 (Depthwise (None, 2, 2, 720) 6480 ['block6a_expand_activation[0]
Conv2D) [0]']
block6a_bn (BatchNormaliza (None, 2, 2, 720) 2880 ['block6a_dwconv2[0][0]']
tion)
block6a_activation (Activa (None, 2, 2, 720) 0 ['block6a_bn[0][0]']
tion)
block6a_se_squeeze (Global (None, 720) 0 ['block6a_activation[0][0]']
AveragePooling2D)
block6a_se_reshape (Reshap (None, 1, 1, 720) 0 ['block6a_se_squeeze[0][0]']
e)
block6a_se_reduce (Conv2D) (None, 1, 1, 30) 21630 ['block6a_se_reshape[0][0]']
block6a_se_expand (Conv2D) (None, 1, 1, 720) 22320 ['block6a_se_reduce[0][0]']
block6a_se_excite (Multipl (None, 2, 2, 720) 0 ['block6a_activation[0][0]',
y) 'block6a_se_expand[0][0]']
block6a_project_conv (Conv (None, 2, 2, 208) 149760 ['block6a_se_excite[0][0]']
2D)
block6a_project_bn (BatchN (None, 2, 2, 208) 832 ['block6a_project_conv[0][0]']
ormalization)
block6b_expand_conv (Conv2 (None, 2, 2, 1248) 259584 ['block6a_project_bn[0][0]']
D)
block6b_expand_bn (BatchNo (None, 2, 2, 1248) 4992 ['block6b_expand_conv[0][0]']
rmalization)
block6b_expand_activation (None, 2, 2, 1248) 0 ['block6b_expand_bn[0][0]']
(Activation)
block6b_dwconv2 (Depthwise (None, 2, 2, 1248) 11232 ['block6b_expand_activation[0]
Conv2D) [0]']
block6b_bn (BatchNormaliza (None, 2, 2, 1248) 4992 ['block6b_dwconv2[0][0]']
tion)
block6b_activation (Activa (None, 2, 2, 1248) 0 ['block6b_bn[0][0]']
tion)
block6b_se_squeeze (Global (None, 1248) 0 ['block6b_activation[0][0]']
AveragePooling2D)
block6b_se_reshape (Reshap (None, 1, 1, 1248) 0 ['block6b_se_squeeze[0][0]']
e)
block6b_se_reduce (Conv2D) (None, 1, 1, 52) 64948 ['block6b_se_reshape[0][0]']
block6b_se_expand (Conv2D) (None, 1, 1, 1248) 66144 ['block6b_se_reduce[0][0]']
block6b_se_excite (Multipl (None, 2, 2, 1248) 0 ['block6b_activation[0][0]',
y) 'block6b_se_expand[0][0]']
block6b_project_conv (Conv (None, 2, 2, 208) 259584 ['block6b_se_excite[0][0]']
2D)
block6b_project_bn (BatchN (None, 2, 2, 208) 832 ['block6b_project_conv[0][0]']
ormalization)
block6b_drop (Dropout) (None, 2, 2, 208) 0 ['block6b_project_bn[0][0]']
block6b_add (Add) (None, 2, 2, 208) 0 ['block6b_drop[0][0]',
'block6a_project_bn[0][0]']
block6c_expand_conv (Conv2 (None, 2, 2, 1248) 259584 ['block6b_add[0][0]']
D)
block6c_expand_bn (BatchNo (None, 2, 2, 1248) 4992 ['block6c_expand_conv[0][0]']
rmalization)
block6c_expand_activation (None, 2, 2, 1248) 0 ['block6c_expand_bn[0][0]']
(Activation)
block6c_dwconv2 (Depthwise (None, 2, 2, 1248) 11232 ['block6c_expand_activation[0]
Conv2D) [0]']
block6c_bn (BatchNormaliza (None, 2, 2, 1248) 4992 ['block6c_dwconv2[0][0]']
tion)
block6c_activation (Activa (None, 2, 2, 1248) 0 ['block6c_bn[0][0]']
tion)
block6c_se_squeeze (Global (None, 1248) 0 ['block6c_activation[0][0]']
AveragePooling2D)
block6c_se_reshape (Reshap (None, 1, 1, 1248) 0 ['block6c_se_squeeze[0][0]']
e)
block6c_se_reduce (Conv2D) (None, 1, 1, 52) 64948 ['block6c_se_reshape[0][0]']
block6c_se_expand (Conv2D) (None, 1, 1, 1248) 66144 ['block6c_se_reduce[0][0]']
block6c_se_excite (Multipl (None, 2, 2, 1248) 0 ['block6c_activation[0][0]',
y) 'block6c_se_expand[0][0]']
block6c_project_conv (Conv (None, 2, 2, 208) 259584 ['block6c_se_excite[0][0]']
2D)
block6c_project_bn (BatchN (None, 2, 2, 208) 832 ['block6c_project_conv[0][0]']
ormalization)
block6c_drop (Dropout) (None, 2, 2, 208) 0 ['block6c_project_bn[0][0]']
block6c_add (Add) (None, 2, 2, 208) 0 ['block6c_drop[0][0]',
'block6b_add[0][0]']
block6d_expand_conv (Conv2 (None, 2, 2, 1248) 259584 ['block6c_add[0][0]']
D)
block6d_expand_bn (BatchNo (None, 2, 2, 1248) 4992 ['block6d_expand_conv[0][0]']
rmalization)
block6d_expand_activation (None, 2, 2, 1248) 0 ['block6d_expand_bn[0][0]']
(Activation)
block6d_dwconv2 (Depthwise (None, 2, 2, 1248) 11232 ['block6d_expand_activation[0]
Conv2D) [0]']
block6d_bn (BatchNormaliza (None, 2, 2, 1248) 4992 ['block6d_dwconv2[0][0]']
tion)
block6d_activation (Activa (None, 2, 2, 1248) 0 ['block6d_bn[0][0]']
tion)
block6d_se_squeeze (Global (None, 1248) 0 ['block6d_activation[0][0]']
AveragePooling2D)
block6d_se_reshape (Reshap (None, 1, 1, 1248) 0 ['block6d_se_squeeze[0][0]']
e)
block6d_se_reduce (Conv2D) (None, 1, 1, 52) 64948 ['block6d_se_reshape[0][0]']
block6d_se_expand (Conv2D) (None, 1, 1, 1248) 66144 ['block6d_se_reduce[0][0]']
block6d_se_excite (Multipl (None, 2, 2, 1248) 0 ['block6d_activation[0][0]',
y) 'block6d_se_expand[0][0]']
block6d_project_conv (Conv (None, 2, 2, 208) 259584 ['block6d_se_excite[0][0]']
2D)
block6d_project_bn (BatchN (None, 2, 2, 208) 832 ['block6d_project_conv[0][0]']
ormalization)
block6d_drop (Dropout) (None, 2, 2, 208) 0 ['block6d_project_bn[0][0]']
block6d_add (Add) (None, 2, 2, 208) 0 ['block6d_drop[0][0]',
'block6c_add[0][0]']
block6e_expand_conv (Conv2 (None, 2, 2, 1248) 259584 ['block6d_add[0][0]']
D)
block6e_expand_bn (BatchNo (None, 2, 2, 1248) 4992 ['block6e_expand_conv[0][0]']
rmalization)
block6e_expand_activation (None, 2, 2, 1248) 0 ['block6e_expand_bn[0][0]']
(Activation)
block6e_dwconv2 (Depthwise (None, 2, 2, 1248) 11232 ['block6e_expand_activation[0]
Conv2D) [0]']
block6e_bn (BatchNormaliza (None, 2, 2, 1248) 4992 ['block6e_dwconv2[0][0]']
tion)
block6e_activation (Activa (None, 2, 2, 1248) 0 ['block6e_bn[0][0]']
tion)
block6e_se_squeeze (Global (None, 1248) 0 ['block6e_activation[0][0]']
AveragePooling2D)
block6e_se_reshape (Reshap (None, 1, 1, 1248) 0 ['block6e_se_squeeze[0][0]']
e)
block6e_se_reduce (Conv2D) (None, 1, 1, 52) 64948 ['block6e_se_reshape[0][0]']
block6e_se_expand (Conv2D) (None, 1, 1, 1248) 66144 ['block6e_se_reduce[0][0]']
block6e_se_excite (Multipl (None, 2, 2, 1248) 0 ['block6e_activation[0][0]',
y) 'block6e_se_expand[0][0]']
block6e_project_conv (Conv (None, 2, 2, 208) 259584 ['block6e_se_excite[0][0]']
2D)
block6e_project_bn (BatchN (None, 2, 2, 208) 832 ['block6e_project_conv[0][0]']
ormalization)
block6e_drop (Dropout) (None, 2, 2, 208) 0 ['block6e_project_bn[0][0]']
block6e_add (Add) (None, 2, 2, 208) 0 ['block6e_drop[0][0]',
'block6d_add[0][0]']
block6f_expand_conv (Conv2 (None, 2, 2, 1248) 259584 ['block6e_add[0][0]']
D)
block6f_expand_bn (BatchNo (None, 2, 2, 1248) 4992 ['block6f_expand_conv[0][0]']
rmalization)
block6f_expand_activation (None, 2, 2, 1248) 0 ['block6f_expand_bn[0][0]']
(Activation)
block6f_dwconv2 (Depthwise (None, 2, 2, 1248) 11232 ['block6f_expand_activation[0]
Conv2D) [0]']
block6f_bn (BatchNormaliza (None, 2, 2, 1248) 4992 ['block6f_dwconv2[0][0]']
tion)
block6f_activation (Activa (None, 2, 2, 1248) 0 ['block6f_bn[0][0]']
tion)
block6f_se_squeeze (Global (None, 1248) 0 ['block6f_activation[0][0]']
AveragePooling2D)
block6f_se_reshape (Reshap (None, 1, 1, 1248) 0 ['block6f_se_squeeze[0][0]']
e)
block6f_se_reduce (Conv2D) (None, 1, 1, 52) 64948 ['block6f_se_reshape[0][0]']
block6f_se_expand (Conv2D) (None, 1, 1, 1248) 66144 ['block6f_se_reduce[0][0]']
block6f_se_excite (Multipl (None, 2, 2, 1248) 0 ['block6f_activation[0][0]',
y) 'block6f_se_expand[0][0]']
block6f_project_conv (Conv (None, 2, 2, 208) 259584 ['block6f_se_excite[0][0]']
2D)
block6f_project_bn (BatchN (None, 2, 2, 208) 832 ['block6f_project_conv[0][0]']
ormalization)
block6f_drop (Dropout) (None, 2, 2, 208) 0 ['block6f_project_bn[0][0]']
block6f_add (Add) (None, 2, 2, 208) 0 ['block6f_drop[0][0]',
'block6e_add[0][0]']
block6g_expand_conv (Conv2 (None, 2, 2, 1248) 259584 ['block6f_add[0][0]']
D)
block6g_expand_bn (BatchNo (None, 2, 2, 1248) 4992 ['block6g_expand_conv[0][0]']
rmalization)
block6g_expand_activation (None, 2, 2, 1248) 0 ['block6g_expand_bn[0][0]']
(Activation)
block6g_dwconv2 (Depthwise (None, 2, 2, 1248) 11232 ['block6g_expand_activation[0]
Conv2D) [0]']
block6g_bn (BatchNormaliza (None, 2, 2, 1248) 4992 ['block6g_dwconv2[0][0]']
tion)
block6g_activation (Activa (None, 2, 2, 1248) 0 ['block6g_bn[0][0]']
tion)
block6g_se_squeeze (Global (None, 1248) 0 ['block6g_activation[0][0]']
AveragePooling2D)
block6g_se_reshape (Reshap (None, 1, 1, 1248) 0 ['block6g_se_squeeze[0][0]']
e)
block6g_se_reduce (Conv2D) (None, 1, 1, 52) 64948 ['block6g_se_reshape[0][0]']
block6g_se_expand (Conv2D) (None, 1, 1, 1248) 66144 ['block6g_se_reduce[0][0]']
block6g_se_excite (Multipl (None, 2, 2, 1248) 0 ['block6g_activation[0][0]',
y) 'block6g_se_expand[0][0]']
block6g_project_conv (Conv (None, 2, 2, 208) 259584 ['block6g_se_excite[0][0]']
2D)
block6g_project_bn (BatchN (None, 2, 2, 208) 832 ['block6g_project_conv[0][0]']
ormalization)
block6g_drop (Dropout) (None, 2, 2, 208) 0 ['block6g_project_bn[0][0]']
block6g_add (Add) (None, 2, 2, 208) 0 ['block6g_drop[0][0]',
'block6f_add[0][0]']
block6h_expand_conv (Conv2 (None, 2, 2, 1248) 259584 ['block6g_add[0][0]']
D)
block6h_expand_bn (BatchNo (None, 2, 2, 1248) 4992 ['block6h_expand_conv[0][0]']
rmalization)
block6h_expand_activation (None, 2, 2, 1248) 0 ['block6h_expand_bn[0][0]']
(Activation)
block6h_dwconv2 (Depthwise (None, 2, 2, 1248) 11232 ['block6h_expand_activation[0]
Conv2D) [0]']
block6h_bn (BatchNormaliza (None, 2, 2, 1248) 4992 ['block6h_dwconv2[0][0]']
tion)
block6h_activation (Activa (None, 2, 2, 1248) 0 ['block6h_bn[0][0]']
tion)
block6h_se_squeeze (Global (None, 1248) 0 ['block6h_activation[0][0]']
AveragePooling2D)
block6h_se_reshape (Reshap (None, 1, 1, 1248) 0 ['block6h_se_squeeze[0][0]']
e)
block6h_se_reduce (Conv2D) (None, 1, 1, 52) 64948 ['block6h_se_reshape[0][0]']
block6h_se_expand (Conv2D) (None, 1, 1, 1248) 66144 ['block6h_se_reduce[0][0]']
block6h_se_excite (Multipl (None, 2, 2, 1248) 0 ['block6h_activation[0][0]',
y) 'block6h_se_expand[0][0]']
block6h_project_conv (Conv (None, 2, 2, 208) 259584 ['block6h_se_excite[0][0]']
2D)
block6h_project_bn (BatchN (None, 2, 2, 208) 832 ['block6h_project_conv[0][0]']
ormalization)
block6h_drop (Dropout) (None, 2, 2, 208) 0 ['block6h_project_bn[0][0]']
block6h_add (Add) (None, 2, 2, 208) 0 ['block6h_drop[0][0]',
'block6g_add[0][0]']
block6i_expand_conv (Conv2 (None, 2, 2, 1248) 259584 ['block6h_add[0][0]']
D)
block6i_expand_bn (BatchNo (None, 2, 2, 1248) 4992 ['block6i_expand_conv[0][0]']
rmalization)
block6i_expand_activation (None, 2, 2, 1248) 0 ['block6i_expand_bn[0][0]']
(Activation)
block6i_dwconv2 (Depthwise (None, 2, 2, 1248) 11232 ['block6i_expand_activation[0]
Conv2D) [0]']
block6i_bn (BatchNormaliza (None, 2, 2, 1248) 4992 ['block6i_dwconv2[0][0]']
tion)
block6i_activation (Activa (None, 2, 2, 1248) 0 ['block6i_bn[0][0]']
tion)
block6i_se_squeeze (Global (None, 1248) 0 ['block6i_activation[0][0]']
AveragePooling2D)
block6i_se_reshape (Reshap (None, 1, 1, 1248) 0 ['block6i_se_squeeze[0][0]']
e)
block6i_se_reduce (Conv2D) (None, 1, 1, 52) 64948 ['block6i_se_reshape[0][0]']
block6i_se_expand (Conv2D) (None, 1, 1, 1248) 66144 ['block6i_se_reduce[0][0]']
block6i_se_excite (Multipl (None, 2, 2, 1248) 0 ['block6i_activation[0][0]',
y) 'block6i_se_expand[0][0]']
block6i_project_conv (Conv (None, 2, 2, 208) 259584 ['block6i_se_excite[0][0]']
2D)
block6i_project_bn (BatchN (None, 2, 2, 208) 832 ['block6i_project_conv[0][0]']
ormalization)
block6i_drop (Dropout) (None, 2, 2, 208) 0 ['block6i_project_bn[0][0]']
block6i_add (Add) (None, 2, 2, 208) 0 ['block6i_drop[0][0]',
'block6h_add[0][0]']
block6j_expand_conv (Conv2 (None, 2, 2, 1248) 259584 ['block6i_add[0][0]']
D)
block6j_expand_bn (BatchNo (None, 2, 2, 1248) 4992 ['block6j_expand_conv[0][0]']
rmalization)
block6j_expand_activation (None, 2, 2, 1248) 0 ['block6j_expand_bn[0][0]']
(Activation)
block6j_dwconv2 (Depthwise (None, 2, 2, 1248) 11232 ['block6j_expand_activation[0]
Conv2D) [0]']
block6j_bn (BatchNormaliza (None, 2, 2, 1248) 4992 ['block6j_dwconv2[0][0]']
tion)
block6j_activation (Activa (None, 2, 2, 1248) 0 ['block6j_bn[0][0]']
tion)
block6j_se_squeeze (Global (None, 1248) 0 ['block6j_activation[0][0]']
AveragePooling2D)
block6j_se_reshape (Reshap (None, 1, 1, 1248) 0 ['block6j_se_squeeze[0][0]']
e)
block6j_se_reduce (Conv2D) (None, 1, 1, 52) 64948 ['block6j_se_reshape[0][0]']
block6j_se_expand (Conv2D) (None, 1, 1, 1248) 66144 ['block6j_se_reduce[0][0]']
block6j_se_excite (Multipl (None, 2, 2, 1248) 0 ['block6j_activation[0][0]',
y) 'block6j_se_expand[0][0]']
block6j_project_conv (Conv (None, 2, 2, 208) 259584 ['block6j_se_excite[0][0]']
2D)
block6j_project_bn (BatchN (None, 2, 2, 208) 832 ['block6j_project_conv[0][0]']
ormalization)
block6j_drop (Dropout) (None, 2, 2, 208) 0 ['block6j_project_bn[0][0]']
block6j_add (Add) (None, 2, 2, 208) 0 ['block6j_drop[0][0]',
'block6i_add[0][0]']
top_conv (Conv2D) (None, 2, 2, 1408) 292864 ['block6j_add[0][0]']
top_bn (BatchNormalization (None, 2, 2, 1408) 5632 ['top_conv[0][0]']
)
top_activation (Activation (None, 2, 2, 1408) 0 ['top_bn[0][0]']
)
==================================================================================================
Total params: 8769374 (33.45 MB)
Trainable params: 8687086 (33.14 MB)
Non-trainable params: 82288 (321.44 KB)
__________________________________________________________________________________________________
# Visualize the model as blocks
plot_model(EfficientNet, to_file='model.png', show_shapes=True, show_layer_names=True)
# Set ResNet layers to non-trainable
EfficientNet.trainable = False
# Rebuild the model
x = EfficientNet.output
# Add Global Average Pooling layer
x = GlobalAveragePooling2D()(x)
# Flatten the output
x = Flatten()(x)
# Add a Dense layer (optional: you can add BatchNormalization and/or Dropout here)
x = Dense(256, activation='relu')(x)
x = BatchNormalization()(x) # Optional
x = Dropout(0.3)(x) # Optional
# Add the final Dense layer with 4 neurons and a softmax activation function
pred = Dense(4, activation='softmax')(x)
# Initializing the model
Efficientnetmodel = Model(inputs=EfficientNet.input, outputs=pred)
# ModelCheckpoint is used to save the model at certain intervals
checkpoint_efnet = ModelCheckpoint("efnet.h5", # file path where the model will be saved
monitor='val_accuracy', # means the callback monitors the validation accuracy
verbose=1, # model is saved only when the metric (val_acc) has improved
save_best_only=True,
mode='max')
# Configured callbacks will be passed to the model during training
callbacks_list_efnet = [early_stopping, checkpoint_efnet, reduce_learningrate]
# Number of iterations over the entire dataset that the training process should run
epochs = 20
# Compile the model (This step is required if you're about to train the model)
Efficientnetmodel.compile(loss='categorical_crossentropy',
optimizer='adam',
metrics=['accuracy'])
# Fitting the model with required parameters above, saving model into history_efnet variable
history_efnet = Efficientnetmodel.fit(
train_set_pre,
steps_per_epoch = train_set_pre.samples // train_set_pre.batch_size,
validation_data = validation_set_pre,
validation_steps = validation_set_pre.samples // validation_set_pre.batch_size,
epochs = 20,
verbose = 1,
callbacks = callbacks_list_efnet
)
Epoch 1/20 472/472 [==============================] - ETA: 0s - loss: 1.4951 - accuracy: 0.2569 Epoch 1: val_accuracy improved from -inf to 0.24435, saving model to efnet.h5 472/472 [==============================] - 27s 52ms/step - loss: 1.4951 - accuracy: 0.2569 - val_loss: 1.7732 - val_accuracy: 0.2444 - lr: 0.0010 Epoch 2/20 472/472 [==============================] - ETA: 0s - loss: 1.4112 - accuracy: 0.2630 Epoch 2: val_accuracy improved from 0.24435 to 0.36673, saving model to efnet.h5 472/472 [==============================] - 24s 52ms/step - loss: 1.4112 - accuracy: 0.2630 - val_loss: 1.3607 - val_accuracy: 0.3667 - lr: 0.0010 Epoch 3/20 471/472 [============================>.] - ETA: 0s - loss: 1.3959 - accuracy: 0.2631 Epoch 3: val_accuracy did not improve from 0.36673 472/472 [==============================] - 23s 50ms/step - loss: 1.3960 - accuracy: 0.2629 - val_loss: 1.3424 - val_accuracy: 0.3665 - lr: 0.0010 Epoch 4/20 471/472 [============================>.] - ETA: 0s - loss: 1.3921 - accuracy: 0.2635 Epoch 4: val_accuracy did not improve from 0.36673 472/472 [==============================] - 23s 49ms/step - loss: 1.3921 - accuracy: 0.2636 - val_loss: 1.3576 - val_accuracy: 0.3163 - lr: 0.0010 Epoch 5/20 472/472 [==============================] - ETA: 0s - loss: 1.3906 - accuracy: 0.2663 Epoch 5: val_accuracy improved from 0.36673 to 0.36734, saving model to efnet.h5 472/472 [==============================] - 24s 50ms/step - loss: 1.3906 - accuracy: 0.2663 - val_loss: 1.3653 - val_accuracy: 0.3673 - lr: 0.0010 Epoch 6/20 471/472 [============================>.] - ETA: 0s - loss: 1.3911 - accuracy: 0.2611Restoring model weights from the end of the best epoch: 3. Epoch 6: val_accuracy did not improve from 0.36734 Epoch 6: ReduceLROnPlateau reducing learning rate to 0.00020000000949949026. 472/472 [==============================] - 23s 50ms/step - loss: 1.3910 - accuracy: 0.2614 - val_loss: 1.3712 - val_accuracy: 0.2599 - lr: 0.0010 Epoch 6: early stopping
# Load the saved model if notebook session gets disconnected
Efficientnetmodel = load_model("efnet.h5")
# Evaluate the model on the test data
test_loss, test_accuracy = Efficientnetmodel.evaluate(test_set_pre, verbose=1)
# Print the results
print("Test Loss:", test_loss)
print("Test Accuracy:", test_accuracy)
4/4 [==============================] - 0s 52ms/step - loss: 1.4318 - accuracy: 0.2500 Test Loss: 1.4318348169326782 Test Accuracy: 0.25
# Plotting the Training and Validation accuracies of EfficientNet model
plt.plot(history_efnet.history['accuracy'])
plt.plot(history_efnet.history['val_accuracy'])
plt.title('EfficientNet Transfer Learning Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc = 'upper left')
# Display the plot
plt.show()
Observations and Insights:
In this section, we will build a more complex Convolutional Neural Network Model that has close to as many parameters as we had in our Transfer Learning Models. However, we will have only 1 input channel for our input images.
In this section, we are creating data loaders which we will use as inputs to the more Complicated Convolutional Neural Network. We will go ahead with color_mode = 'grayscale'.
# Clearing backend
backend.clear_session()
# Reset seed for number generators using reset_seed() function
reset_seed(fixed_seed)
Random number generators seed has been set to 42
# Creating dataloader for train for Greyscale dataset
train_set_gray2 = datagen.flow_from_directory(folder_path + "train",
target_size = (img_size),
color_mode = 'grayscale',
batch_size = batch_size,
class_mode = 'categorical',
classes = ['happy', 'sad', 'neutral', 'surprise'],
shuffle = True)
# Creating dataloader for validation for Greyscale dataset
validation_set_gray2 = datagen.flow_from_directory(folder_path + "validation",
target_size = (img_size),
color_mode = 'grayscale',
batch_size = batch_size,
class_mode = 'categorical',
classes = ['happy', 'sad', 'neutral', 'surprise'],
shuffle = True)
# Creating dataloader for test for Greyscale dataset
test_set_gray2 = datagen.flow_from_directory(folder_path + "test",
target_size = (img_size),
color_mode = 'grayscale',
batch_size = batch_size,
class_mode = 'categorical',
classes = ['happy', 'sad', 'neutral', 'surprise'],
shuffle = True)
Found 15109 images belonging to 4 classes. Found 4977 images belonging to 4 classes. Found 128 images belonging to 4 classes.
from keras.layers import Dense, BatchNormalization, Activation, Dropout
# Define more complex CNN model
def complex_cnn():
cnn_model = Sequential()
# Building a layer with 5 Convolutional Blocks and see if performance increases
# Add first Conv2D layer with 64 filters and a kernel size of 2. Use the 'same'
# padding and provide the input shape = (48, 48, 1). Use 'relu' activation.
cnn_model.add(Conv2D(filters = 64,
kernel_size = (2, 2),
padding = "same",
input_shape = (img_size[0], img_size[1], 1), activation = 'relu'))
# Adding a BatchNormalization layer
cnn_model.add(BatchNormalization())
# LeakyRelU layer with Leaky ReLU parameter of 0.1
cnn_model.add(LeakyReLU(0.1))
# A max-pooling layer with a pool size of 2x2
cnn_model.add(MaxPooling2D(pool_size = (2, 2)))
# Dropout layer with the rate equal to 0.2
cnn_model.add(Dropout(0.2))
# Second Convolutional layer with 128 filters and the kernel size of 2x2 with 'same' padding
cnn_model.add(Conv2D(filters = 128, kernel_size = (2, 2), padding = "same", activation = 'relu'))
# Adding a BatchNormalization layer
cnn_model.add(BatchNormalization())
# LeakyRelU layer with Leaky ReLU parameter of 0.1
cnn_model.add(LeakyReLU(0.1))
# A max-pooling layer with a pool size of 2x2
cnn_model.add(MaxPooling2D(pool_size = (2, 2)))
# Dropout layer with the rate equal to 0.2
cnn_model.add(Dropout(0.2))
# Third Convolutional layer with 512 filters and the kernel size of 2x2 with 'same' padding
cnn_model.add(Conv2D(filters = 512, kernel_size = (2, 2), padding = "same", activation = 'relu'))
# Adding a BatchNormalization layer
cnn_model.add(BatchNormalization())
# LeakyRelU layer with Leaky ReLU parameter of 0.1
cnn_model.add(LeakyReLU(0.1))
# A max-pooling layer with a pool size of 2x2
cnn_model.add(MaxPooling2D(pool_size = (2, 2)))
# Dropout layer with the rate equal to 0.2
cnn_model.add(Dropout(0.2))
# Fourth Convolutional layer with 512 filters and the kernel size of 2x2 with 'same' padding
cnn_model.add(Conv2D(filters = 512, kernel_size = (2, 2), padding = "same", activation = 'relu'))
# Adding a BatchNormalization layer
cnn_model.add(BatchNormalization())
# LeakyRelU layer with Leaky ReLU parameter of 0.1
cnn_model.add(LeakyReLU(0.1))
# A max-pooling layer with a pool size of 2x2
cnn_model.add(MaxPooling2D(pool_size = (2, 2)))
# Dropout layer with the rate equal to 0.2
cnn_model.add(Dropout(0.2))
# Fifth Convolutional layer with 128 filters and the kernel size of 2x2 with 'same' padding
cnn_model.add(Conv2D(filters = 128, kernel_size = (2, 2), padding = "same", activation = 'relu'))
# Adding a BatchNormalization layer
cnn_model.add(BatchNormalization())
# LeakyRelU layer with Leaky ReLU parameter of 0.1
cnn_model.add(LeakyReLU(0.1))
# A max-pooling layer with a pool size of 2x2
cnn_model.add(MaxPooling2D(pool_size = (2, 2)))
# Dropout layer with the rate equal to 0.2
cnn_model.add(Dropout(0.2))
# Flatten the output from the previous layer
cnn_model.add(Flatten())
# Dense layer with 256 nodes
cnn_model.add(Dense(256, activation = 'relu'))
# Adding a BatchNormalization layer
cnn_model.add(BatchNormalization())
# Adding a 'relu' Activation layer
cnn_model.add(Activation('relu'))
# Dropout layer with the rate equal to 0.2
cnn_model.add(Dropout(0.2))
# Dense layer with 512 nodes
cnn_model.add(Dense(512, activation = 'relu'))
# Adding a BatchNormalization layer
cnn_model.add(BatchNormalization())
# Adding a 'relu' Activation layer
cnn_model.add(Activation('relu'))
# Dropout layer with the rate equal to 0.2
cnn_model.add(Dropout(0.2))
# Final output layer with nodes equal to the number of classes (sad, happy, neutral and surprised) and 'softmax' as the activation function
cnn_model.add(Dense(4, activation = 'softmax'))
# Compiling the CNN model with categorical crossentropy as loss function, Adam Optimizer
# with 0.001 learning rate, and set metrics set to 'accuracy'.
cnn_model.compile(loss='categorical_crossentropy',
optimizer=legacy.Adam(learning_rate = 0.001),
metrics=['accuracy'])
return cnn_model
# Building the model
complex_cnn_model = complex_cnn()
# Print first model's summary
complex_cnn_model.summary()
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 48, 48, 64) 320
batch_normalization (Batch (None, 48, 48, 64) 256
Normalization)
leaky_re_lu (LeakyReLU) (None, 48, 48, 64) 0
max_pooling2d (MaxPooling2 (None, 24, 24, 64) 0
D)
dropout (Dropout) (None, 24, 24, 64) 0
conv2d_1 (Conv2D) (None, 24, 24, 128) 32896
batch_normalization_1 (Bat (None, 24, 24, 128) 512
chNormalization)
leaky_re_lu_1 (LeakyReLU) (None, 24, 24, 128) 0
max_pooling2d_1 (MaxPoolin (None, 12, 12, 128) 0
g2D)
dropout_1 (Dropout) (None, 12, 12, 128) 0
conv2d_2 (Conv2D) (None, 12, 12, 512) 262656
batch_normalization_2 (Bat (None, 12, 12, 512) 2048
chNormalization)
leaky_re_lu_2 (LeakyReLU) (None, 12, 12, 512) 0
max_pooling2d_2 (MaxPoolin (None, 6, 6, 512) 0
g2D)
dropout_2 (Dropout) (None, 6, 6, 512) 0
conv2d_3 (Conv2D) (None, 6, 6, 512) 1049088
batch_normalization_3 (Bat (None, 6, 6, 512) 2048
chNormalization)
leaky_re_lu_3 (LeakyReLU) (None, 6, 6, 512) 0
max_pooling2d_3 (MaxPoolin (None, 3, 3, 512) 0
g2D)
dropout_3 (Dropout) (None, 3, 3, 512) 0
conv2d_4 (Conv2D) (None, 3, 3, 128) 262272
batch_normalization_4 (Bat (None, 3, 3, 128) 512
chNormalization)
leaky_re_lu_4 (LeakyReLU) (None, 3, 3, 128) 0
max_pooling2d_4 (MaxPoolin (None, 1, 1, 128) 0
g2D)
dropout_4 (Dropout) (None, 1, 1, 128) 0
flatten (Flatten) (None, 128) 0
dense (Dense) (None, 256) 33024
batch_normalization_5 (Bat (None, 256) 1024
chNormalization)
activation (Activation) (None, 256) 0
dropout_5 (Dropout) (None, 256) 0
dense_1 (Dense) (None, 512) 131584
batch_normalization_6 (Bat (None, 512) 2048
chNormalization)
activation_1 (Activation) (None, 512) 0
dropout_6 (Dropout) (None, 512) 0
dense_2 (Dense) (None, 4) 2052
=================================================================
Total params: 1782340 (6.80 MB)
Trainable params: 1778116 (6.78 MB)
Non-trainable params: 4224 (16.50 KB)
_________________________________________________________________
# Visualize the model as blocks
plot_model(complex_cnn_model, to_file='model.png', show_shapes=True, show_layer_names=True)
# ModelCheckpoint is used to save the model at certain intervals
checkpoint_complex_cnn = ModelCheckpoint("complex_cnn.h5", # file path where the model will be saved
monitor='val_accuracy', # means the callback monitors the validation accuracy
verbose=1, # model is saved only when the metric (val_acc) has improved
save_best_only=True,
mode='max')
# Configured callbacks will be passed to the model during training
callbacks_list_complex_cnn = [early_stopping, checkpoint_complex_cnn, reduce_learningrate]
# Number of iterations over the entire dataset that the training process should run
epochs = 35
# Compiling the Complex CNN model with categorical crossentropy as loss function, Adam Optimizer
# with 0.003 learning rate, and set metrics set to 'accuracy'.
complex_cnn_model.compile(loss='categorical_crossentropy',
optimizer=legacy.Adam(learning_rate = 0.003),
metrics=['accuracy'])
# Fitting the model with required parameters above, saving model into history_complex_cnn variable
history_complex_cnn = complex_cnn_model.fit(
train_set_gray2,
steps_per_epoch = train_set_gray2.samples // train_set_gray2.batch_size,
validation_data = validation_set_gray2,
validation_steps = validation_set_gray2.samples // validation_set_gray2.batch_size,
epochs = 20,
verbose = 1,
callbacks = callbacks_list_complex_cnn
)
Epoch 1/20 472/472 [==============================] - ETA: 0s - loss: 1.5206 - accuracy: 0.3041 Epoch 1: val_accuracy improved from -inf to 0.22903, saving model to complex_cnn.h5 472/472 [==============================] - 1672s 4s/step - loss: 1.5206 - accuracy: 0.3041 - val_loss: 1.4122 - val_accuracy: 0.2290 - lr: 0.0030 Epoch 2/20 472/472 [==============================] - ETA: 0s - loss: 1.3079 - accuracy: 0.3838 Epoch 2: val_accuracy improved from 0.22903 to 0.38387, saving model to complex_cnn.h5 472/472 [==============================] - 1661s 4s/step - loss: 1.3079 - accuracy: 0.3838 - val_loss: 1.6416 - val_accuracy: 0.3839 - lr: 0.0030 Epoch 3/20 472/472 [==============================] - ETA: 0s - loss: 1.2012 - accuracy: 0.4497 Epoch 3: val_accuracy did not improve from 0.38387 472/472 [==============================] - 1687s 4s/step - loss: 1.2012 - accuracy: 0.4497 - val_loss: 1.4566 - val_accuracy: 0.3149 - lr: 0.0030 Epoch 4/20 472/472 [==============================] - ETA: 0s - loss: 1.1247 - accuracy: 0.4902 Epoch 4: val_accuracy improved from 0.38387 to 0.48589, saving model to complex_cnn.h5 472/472 [==============================] - 1657s 4s/step - loss: 1.1247 - accuracy: 0.4902 - val_loss: 1.1265 - val_accuracy: 0.4859 - lr: 0.0030 Epoch 5/20 472/472 [==============================] - ETA: 0s - loss: 1.0590 - accuracy: 0.5402 Epoch 5: val_accuracy improved from 0.48589 to 0.54657, saving model to complex_cnn.h5 472/472 [==============================] - 1650s 3s/step - loss: 1.0590 - accuracy: 0.5402 - val_loss: 1.0444 - val_accuracy: 0.5466 - lr: 0.0030 Epoch 6/20 472/472 [==============================] - ETA: 0s - loss: 1.0188 - accuracy: 0.5548 Epoch 6: val_accuracy did not improve from 0.54657 472/472 [==============================] - 1628s 3s/step - loss: 1.0188 - accuracy: 0.5548 - val_loss: 1.7051 - val_accuracy: 0.4889 - lr: 0.0030 Epoch 7/20 472/472 [==============================] - ETA: 0s - loss: 0.9788 - accuracy: 0.5744 Epoch 7: val_accuracy improved from 0.54657 to 0.56754, saving model to complex_cnn.h5 472/472 [==============================] - 1624s 3s/step - loss: 0.9788 - accuracy: 0.5744 - val_loss: 0.9949 - val_accuracy: 0.5675 - lr: 0.0030 Epoch 8/20 472/472 [==============================] - ETA: 0s - loss: 0.9385 - accuracy: 0.5986 Epoch 8: val_accuracy improved from 0.56754 to 0.64718, saving model to complex_cnn.h5 472/472 [==============================] - 1662s 4s/step - loss: 0.9385 - accuracy: 0.5986 - val_loss: 1.2303 - val_accuracy: 0.6472 - lr: 0.0030 Epoch 9/20 472/472 [==============================] - ETA: 0s - loss: 0.9204 - accuracy: 0.6058 Epoch 9: val_accuracy did not improve from 0.64718 472/472 [==============================] - 1563s 3s/step - loss: 0.9204 - accuracy: 0.6058 - val_loss: 1.7770 - val_accuracy: 0.5704 - lr: 0.0030 Epoch 10/20 472/472 [==============================] - ETA: 0s - loss: 0.8911 - accuracy: 0.6192 Epoch 10: val_accuracy improved from 0.64718 to 0.64879, saving model to complex_cnn.h5 472/472 [==============================] - 1527s 3s/step - loss: 0.8911 - accuracy: 0.6192 - val_loss: 0.8410 - val_accuracy: 0.6488 - lr: 0.0030 Epoch 11/20 472/472 [==============================] - ETA: 0s - loss: 0.8497 - accuracy: 0.6406 Epoch 11: val_accuracy did not improve from 0.64879 472/472 [==============================] - 1531s 3s/step - loss: 0.8497 - accuracy: 0.6406 - val_loss: 1.2406 - val_accuracy: 0.5897 - lr: 0.0030 Epoch 12/20 472/472 [==============================] - ETA: 0s - loss: 0.8294 - accuracy: 0.6520 Epoch 12: val_accuracy improved from 0.64879 to 0.65706, saving model to complex_cnn.h5 472/472 [==============================] - 1544s 3s/step - loss: 0.8294 - accuracy: 0.6520 - val_loss: 0.8466 - val_accuracy: 0.6571 - lr: 0.0030 Epoch 13/20 472/472 [==============================] - ETA: 0s - loss: 0.7901 - accuracy: 0.6725 Epoch 13: val_accuracy improved from 0.65706 to 0.67823, saving model to complex_cnn.h5 472/472 [==============================] - 1531s 3s/step - loss: 0.7901 - accuracy: 0.6725 - val_loss: 0.8096 - val_accuracy: 0.6782 - lr: 0.0030 Epoch 14/20 472/472 [==============================] - ETA: 0s - loss: 0.8355 - accuracy: 0.6467 Epoch 14: val_accuracy did not improve from 0.67823 472/472 [==============================] - 1540s 3s/step - loss: 0.8355 - accuracy: 0.6467 - val_loss: 1.5303 - val_accuracy: 0.5716 - lr: 0.0030 Epoch 15/20 472/472 [==============================] - ETA: 0s - loss: 0.8029 - accuracy: 0.6600 Epoch 15: val_accuracy did not improve from 0.67823 472/472 [==============================] - 1517s 3s/step - loss: 0.8029 - accuracy: 0.6600 - val_loss: 1.1180 - val_accuracy: 0.6544 - lr: 0.0030 Epoch 16/20 472/472 [==============================] - ETA: 0s - loss: 0.8643 - accuracy: 0.6320Restoring model weights from the end of the best epoch: 13. Epoch 16: val_accuracy did not improve from 0.67823 Epoch 16: ReduceLROnPlateau reducing learning rate to 0.0006000000052154065. 472/472 [==============================] - 1523s 3s/step - loss: 0.8643 - accuracy: 0.6320 - val_loss: 0.8522 - val_accuracy: 0.6417 - lr: 0.0030 Epoch 16: early stopping
# Load the saved model if notebook session gets disconnected
complex_cnn_model = load_model("complex_cnn.h5")
# Print first model's summary
complex_cnn_model.summary()
Model: "sequential_1"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d_5 (Conv2D) (None, 224, 224, 64) 320
batch_normalization_7 (Bat (None, 224, 224, 64) 256
chNormalization)
leaky_re_lu_5 (LeakyReLU) (None, 224, 224, 64) 0
max_pooling2d_5 (MaxPoolin (None, 112, 112, 64) 0
g2D)
dropout_7 (Dropout) (None, 112, 112, 64) 0
conv2d_6 (Conv2D) (None, 112, 112, 128) 32896
batch_normalization_8 (Bat (None, 112, 112, 128) 512
chNormalization)
leaky_re_lu_6 (LeakyReLU) (None, 112, 112, 128) 0
max_pooling2d_6 (MaxPoolin (None, 56, 56, 128) 0
g2D)
dropout_8 (Dropout) (None, 56, 56, 128) 0
conv2d_7 (Conv2D) (None, 56, 56, 512) 262656
batch_normalization_9 (Bat (None, 56, 56, 512) 2048
chNormalization)
leaky_re_lu_7 (LeakyReLU) (None, 56, 56, 512) 0
max_pooling2d_7 (MaxPoolin (None, 28, 28, 512) 0
g2D)
dropout_9 (Dropout) (None, 28, 28, 512) 0
conv2d_8 (Conv2D) (None, 28, 28, 512) 1049088
batch_normalization_10 (Ba (None, 28, 28, 512) 2048
tchNormalization)
leaky_re_lu_8 (LeakyReLU) (None, 28, 28, 512) 0
max_pooling2d_8 (MaxPoolin (None, 14, 14, 512) 0
g2D)
dropout_10 (Dropout) (None, 14, 14, 512) 0
conv2d_9 (Conv2D) (None, 14, 14, 128) 262272
batch_normalization_11 (Ba (None, 14, 14, 128) 512
tchNormalization)
leaky_re_lu_9 (LeakyReLU) (None, 14, 14, 128) 0
max_pooling2d_9 (MaxPoolin (None, 7, 7, 128) 0
g2D)
dropout_11 (Dropout) (None, 7, 7, 128) 0
flatten_1 (Flatten) (None, 6272) 0
dense_3 (Dense) (None, 256) 1605888
batch_normalization_12 (Ba (None, 256) 1024
tchNormalization)
activation_2 (Activation) (None, 256) 0
dropout_12 (Dropout) (None, 256) 0
dense_4 (Dense) (None, 512) 131584
batch_normalization_13 (Ba (None, 512) 2048
tchNormalization)
activation_3 (Activation) (None, 512) 0
dropout_13 (Dropout) (None, 512) 0
dense_5 (Dense) (None, 4) 2052
=================================================================
Total params: 3355204 (12.80 MB)
Trainable params: 3350980 (12.78 MB)
Non-trainable params: 4224 (16.50 KB)
_________________________________________________________________
# Evaluate the model on the test data
test_loss, test_accuracy = complex_cnn_model.evaluate(test_set_gray2, verbose=1)
# Print the results
print("Test Loss:", test_loss)
print("Test Accuracy:", test_accuracy)
4/4 [==============================] - 1s 158ms/step - loss: 1.2255 - accuracy: 0.4453 Test Loss: 1.225456714630127 Test Accuracy: 0.4453125
# Plotting the Training and Validation accuracies of EfficientNet model
plt.plot(history_complex_cnn.history['accuracy'])
plt.plot(history_complex_cnn.history['val_accuracy'])
plt.title('Complex CNN Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc = 'upper left')
# Display the plot
plt.show()
Observations and Insights:
# Plot the confusion matrix and generate a classification report for the model
from sklearn.metrics import classification_report
from sklearn.metrics import confusion_matrix
test_set = datagen.flow_from_directory(folder_path + "test",
target_size = (img_size[0],img_size[1]),
color_mode = 'grayscale',
batch_size = 128,
class_mode = 'categorical',
classes = ['happy', 'sad', 'neutral', 'surprise'],
shuffle = True)
test_images, test_labels = next(test_set)
# Evaluating with model2_gray
pred = model2_gray.predict(test_images)
pred = np.argmax(pred, axis = 1)
y_true = np.argmax(test_labels, axis = 1)
# Printing the classification report
print(classification_report(y_true, pred, target_names=['happy', 'sad', 'neutral', 'surprise']))
# Plotting the heatmap using confusion matrix
cm = confusion_matrix(y_true, pred)
plt.figure(figsize = (8, 5))
sns.heatmap(cm, annot = True,
fmt = '.0f',
xticklabels = ['happy', 'sad', 'neutral', 'surprise'],
yticklabels = ['happy', 'sad', 'neutral', 'surprise'])
plt.ylabel('Actual')
plt.xlabel('Predicted')
plt.show()
Found 128 images belonging to 4 classes.
4/4 [==============================] - 0s 46ms/step
precision recall f1-score support
happy 0.59 0.72 0.65 32
sad 0.23 0.22 0.23 32
neutral 0.30 0.28 0.29 32
surprise 0.83 0.75 0.79 32
accuracy 0.49 128
macro avg 0.49 0.49 0.49 128
weighted avg 0.49 0.49 0.49 128
Observations and Insights:
import tensorflow_hub as hub
import torch
import torch.nn as nn
import torchvision.models as models
import torch.optim as optim
from torchvision import transforms
from torch.utils.data import DataLoader
from torchvision.datasets import ImageFolder
# Create an instance of ImageDataGenerator with additional image transformations (Horizontal Flip and rotation)
datagen = ImageDataGenerator(
rescale = 1. / 255,
horizontal_flip = True,
vertical_flip = False,
rotation_range = 25,
zoom_range=0.5,
brightness_range=(0.,2.),
shear_range=0.3)
# Set the target size to 224x224
img_size = (224, 224)
print("Images will be scaled to " + str(img_size) + " pixels")
# Creating dataloader for train for RGB dataset
train_set_pre = datagen.flow_from_directory(folder_path + "train",
target_size = (img_size),
color_mode = 'rgb',
batch_size = batch_size,
class_mode = 'categorical',
classes = ['happy', 'sad', 'neutral', 'surprise'],
shuffle = True)
# Creating dataloader for validation for RGB dataset
validation_set_pre = datagen.flow_from_directory(folder_path + "validation",
target_size = (img_size),
color_mode = 'rgb',
batch_size = batch_size,
class_mode = 'categorical',
classes = ['happy', 'sad', 'neutral', 'surprise'],
shuffle = True)
# Creating dataloader for test for RGB dataset
test_set_pre = datagen.flow_from_directory(folder_path + "test",
target_size = (img_size),
color_mode = 'rgb',
batch_size = batch_size,
class_mode = 'categorical',
classes = ['happy', 'sad', 'neutral', 'surprise'],
shuffle = True)
Images will be scaled to (224, 224) pixels Found 15109 images belonging to 4 classes. Found 4977 images belonging to 4 classes. Found 128 images belonging to 4 classes.
# Clearing backend
backend.clear_session()
# Reset seed for number generators using reset_seed() function
reset_seed(fixed_seed)
Random number generators seed has been set to 42
# Importing Weights and Biases library and creating new project
"""
import wandb
from wandb.keras import WandbCallback
wandb.init(project="fine-tune mobilenetv3")
"""
'\nimport wandb\nfrom wandb.keras import WandbCallback\nwandb.init(project="fine-tune mobilenetv3")\n'
# mobilenetv3 will contain the imported model
mobilenetv3 = tf.keras.Sequential([
hub.KerasLayer("https://www.kaggle.com/models/google/mobilenet-v3/frameworks/TensorFlow2/variations/large-075-224-feature-vector/versions/1",
trainable=False), # Can be True, see below.
tf.keras.layers.Dense(4, activation='softmax')
])
# Batch input shape with scaled images (from 48x48 to 224x224 pixels)
mobilenetv3.build([None, img_size[0], img_size[1], 3])
mobilenetv3.summary()
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
keras_layer (KerasLayer) (None, 1280) 2731616
dense (Dense) (None, 4) 5124
=================================================================
Total params: 2736740 (10.44 MB)
Trainable params: 5124 (20.02 KB)
Non-trainable params: 2731616 (10.42 MB)
_________________________________________________________________
# Visualize the model as blocks
from tensorflow.keras.utils import plot_model
plot_model(mobilenetv3, to_file='model.png', show_shapes=True, show_layer_names=True)
# ModelCheckpoint is used to save the model at certain intervals
checkpoint_mobnetv3 = ModelCheckpoint("mobilenetv3.h5", # file path where the model will be saved
monitor='val_accuracy', # means the callback monitors the validation accuracy
verbose=1, # model is saved only when the metric (val_acc) has improved
save_best_only=True,
mode='max')
# Stop the training process early if a monitored metric has stopped improving
early_stopping = EarlyStopping(monitor = 'val_loss', # monitors the validation loss
min_delta = 0, # any positive change is considered as an improvement
patience = 3, # training will be stopped if no improvement is seen for 3 epochs
verbose = 1,
restore_best_weights = True # weights are reverted to best value after early stopping
)
# Reduces the learning rate when a metric has stopped improving
reduce_learningrate = ReduceLROnPlateau(monitor = 'val_loss', # monitor the validation loss
factor = 0.2, # learning rate reduced to 20% when reduction is triggered
patience = 3, # reduction happens after 3 epochs with no improvement
verbose = 1,
min_delta = 0.0001)
# Configured callbacks will be passed to the model during training
callbacks_list_mobnetv3 = [early_stopping, checkpoint_mobnetv3, reduce_learningrate]
# Number of iterations over the entire dataset that the training process should run
epochs = 20
# Compiling the CNN model with categorical crossentropy as loss function, Adam Optimizer
# with 0.01 learning rate, and set metrics set to 'accuracy'.
mobilenetv3.compile(loss='categorical_crossentropy',
optimizer=legacy.Adam(learning_rate = 0.01),
metrics=['accuracy'])
total_sample=train_set_pre.n
# Fitting the model with required parameters above, saving model into history_resnet variable
history_mobilenetv3 = mobilenetv3.fit(
train_set_pre,
steps_per_epoch = train_set_pre.samples // train_set_pre.batch_size,
validation_data = validation_set_pre,
validation_steps = validation_set_pre.samples // validation_set_pre.batch_size,
epochs = epochs,
verbose = 1,
callbacks = callbacks_list_mobnetv3
)
Epoch 1/20 472/472 [==============================] - ETA: 0s - loss: 1.4213 - accuracy: 0.4216 Epoch 1: val_accuracy improved from -inf to 0.42480, saving model to mobilenetv3.h5 472/472 [==============================] - 162s 339ms/step - loss: 1.4213 - accuracy: 0.4216 - val_loss: 1.4765 - val_accuracy: 0.4248 - lr: 0.0100 Epoch 2/20
/Users/ceb/anaconda3/lib/python3.11/site-packages/keras/src/engine/training.py:3103: UserWarning: You are saving your model as an HDF5 file via `model.save()`. This file format is considered legacy. We recommend using instead the native Keras format, e.g. `model.save('my_model.keras')`.
saving_api.save_model(
472/472 [==============================] - ETA: 0s - loss: 1.3875 - accuracy: 0.4533 Epoch 2: val_accuracy improved from 0.42480 to 0.45101, saving model to mobilenetv3.h5 472/472 [==============================] - 191s 404ms/step - loss: 1.3875 - accuracy: 0.4533 - val_loss: 1.3379 - val_accuracy: 0.4510 - lr: 0.0100 Epoch 3/20 472/472 [==============================] - ETA: 0s - loss: 1.4253 - accuracy: 0.4483 Epoch 3: val_accuracy improved from 0.45101 to 0.48206, saving model to mobilenetv3.h5 472/472 [==============================] - 205s 434ms/step - loss: 1.4253 - accuracy: 0.4483 - val_loss: 1.3701 - val_accuracy: 0.4821 - lr: 0.0100 Epoch 4/20 472/472 [==============================] - ETA: 0s - loss: 1.3845 - accuracy: 0.4584 Epoch 4: val_accuracy improved from 0.48206 to 0.48831, saving model to mobilenetv3.h5 472/472 [==============================] - 193s 408ms/step - loss: 1.3845 - accuracy: 0.4584 - val_loss: 1.3222 - val_accuracy: 0.4883 - lr: 0.0100 Epoch 5/20 472/472 [==============================] - ETA: 0s - loss: 1.3962 - accuracy: 0.4552 Epoch 5: val_accuracy did not improve from 0.48831 472/472 [==============================] - 208s 441ms/step - loss: 1.3962 - accuracy: 0.4552 - val_loss: 1.5967 - val_accuracy: 0.4006 - lr: 0.0100 Epoch 6/20 472/472 [==============================] - ETA: 0s - loss: 1.3959 - accuracy: 0.4589 Epoch 6: val_accuracy improved from 0.48831 to 0.49516, saving model to mobilenetv3.h5 472/472 [==============================] - 162s 343ms/step - loss: 1.3959 - accuracy: 0.4589 - val_loss: 1.3347 - val_accuracy: 0.4952 - lr: 0.0100 Epoch 7/20 472/472 [==============================] - ETA: 0s - loss: 1.4257 - accuracy: 0.4575Restoring model weights from the end of the best epoch: 4. Epoch 7: val_accuracy did not improve from 0.49516 Epoch 7: ReduceLROnPlateau reducing learning rate to 0.0019999999552965165. 472/472 [==============================] - 173s 367ms/step - loss: 1.4257 - accuracy: 0.4575 - val_loss: 1.3266 - val_accuracy: 0.4887 - lr: 0.0100 Epoch 7: early stopping
# Clearing backend
backend.clear_session()
# Reset seed for number generators using reset_seed() function
reset_seed(fixed_seed)
Random number generators seed has been set to 42
# Define MobileNetV3 as a feature extractor
mobilenet_v3 = hub.KerasLayer("https://tfhub.dev/google/imagenet/mobilenet_v3_large_075_224/feature_vector/5",
trainable=False)
mnetv3_2 = tf.keras.Sequential([
# Feature extractor
mobilenet_v3,
# Adding Dense layer with 128 nodes and "relu" activation
tf.keras.layers.Dense(128, activation="relu"),
# Adding dropout layer
tf.keras.layers.Dropout(0.5),
# BatchNormalisation layer
tf.keras.layers.BatchNormalization(),
# Flattening all layers
tf.keras.layers.Flatten(),
# Output layer with 4 classes and "softmax" activation
tf.keras.layers.Dense(4, activation="softmax")
])
# Batch input shape with scaled images (from 48x48 to 224x224 pixels)
mnetv3_2.build([None, img_size[0], img_size[1], 3])
# Define MobileNetV3 as a feature extractor
mobilenet_v3 = hub.KerasLayer("https://tfhub.dev/google/imagenet/mobilenet_v3_large_075_224/feature_vector/5",
trainable=False,input_shape=(224,224,3))
class SEBlock(tf.keras.layers.Layer):
def __init__(self, ratio=16):
super(SEBlock, self).__init__()
self.ratio = ratio
# Removing the GlobalAveragePooling2D layer since it's not needed for 2D inputs
def build(self, input_shape):
num_channels = input_shape[-1]
self.fc1 = tf.keras.layers.Dense(num_channels // self.ratio, activation='relu')
self.fc2 = tf.keras.layers.Dense(num_channels, activation='sigmoid')
def call(self, inputs):
# Using `inputs` instead of `x`
x = self.fc1(inputs)
x = self.fc2(x)
return x * inputs
# Create and build the model
target_size = (224, 224)
model = tf.keras.Sequential([
mobilenet_v3,
SEBlock(ratio=16),
tf.keras.layers.Dense(128, activation="relu"),
tf.keras.layers.Dropout(0.5),
tf.keras.layers.BatchNormalization(),
tf.keras.layers.Flatten(),
tf.keras.layers.Dense(4, activation="softmax")
])
model.build((32, *target_size, 3)) # Define the input shape
# Compiling the CNN model with categorical crossentropy as loss function, Adam Optimizer
# with 0.01 learning rate, and set metrics set to 'accuracy'.
model.compile(loss='categorical_crossentropy',
optimizer=legacy.Adam(learning_rate = 0.01),
metrics=['accuracy'])
total_sample=train_set_pre.n
#Model summary
mnetv3_2.summary()
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
keras_layer (KerasLayer) (None, 1280) 2731616
dense (Dense) (None, 128) 163968
dropout (Dropout) (None, 128) 0
batch_normalization (Batch (None, 128) 512
Normalization)
flatten (Flatten) (None, 128) 0
dense_1 (Dense) (None, 4) 516
=================================================================
Total params: 2896612 (11.05 MB)
Trainable params: 164740 (643.52 KB)
Non-trainable params: 2731872 (10.42 MB)
_________________________________________________________________
# ModelCheckpoint is used to save the model at certain intervals
checkpoint_mobnetv3_1 = ModelCheckpoint("mobilenetv3_1.h5", # file path where the model will be saved
monitor='val_accuracy', # means the callback monitors the validation accuracy
verbose=1, # model is saved only when the metric (val_acc) has improved
save_best_only=True,
mode='max')
# Configured callbacks will be passed to the model during training
callbacks_list_mobnetv3 = [early_stopping, checkpoint_mobnetv3_1, reduce_learningrate]
# Number of iterations over the entire dataset that the training process should run
epochs = 20
# Fitting the model with required parameters above, saving model into history_resnet variable
history_mobilenetv3_1 = model.fit(
train_set_pre,
steps_per_epoch = train_set_pre.samples // train_set_pre.batch_size,
validation_data = validation_set_pre,
validation_steps = validation_set_pre.samples // validation_set_pre.batch_size,
epochs = epochs,
verbose = 1,
callbacks = callbacks_list_mobnetv3
)
Epoch 1/20 472/472 [==============================] - ETA: 0s - loss: 1.3299 - accuracy: 0.3697 Epoch 1: val_accuracy improved from -inf to 0.43871, saving model to mobilenetv3_1.h5 472/472 [==============================] - 167s 352ms/step - loss: 1.3299 - accuracy: 0.3697 - val_loss: 1.2176 - val_accuracy: 0.4387 - lr: 0.0100 Epoch 2/20 472/472 [==============================] - ETA: 0s - loss: 1.2501 - accuracy: 0.4137 Epoch 2: val_accuracy improved from 0.43871 to 0.44375, saving model to mobilenetv3_1.h5 472/472 [==============================] - 160s 339ms/step - loss: 1.2501 - accuracy: 0.4137 - val_loss: 1.2040 - val_accuracy: 0.4437 - lr: 0.0100 Epoch 3/20 472/472 [==============================] - ETA: 0s - loss: 1.2461 - accuracy: 0.4222 Epoch 3: val_accuracy did not improve from 0.44375 472/472 [==============================] - 164s 347ms/step - loss: 1.2461 - accuracy: 0.4222 - val_loss: 1.2318 - val_accuracy: 0.4187 - lr: 0.0100 Epoch 4/20 472/472 [==============================] - ETA: 0s - loss: 1.2290 - accuracy: 0.4295 Epoch 4: val_accuracy did not improve from 0.44375 472/472 [==============================] - 196s 414ms/step - loss: 1.2290 - accuracy: 0.4295 - val_loss: 1.1923 - val_accuracy: 0.4347 - lr: 0.0100 Epoch 5/20 472/472 [==============================] - ETA: 0s - loss: 1.2213 - accuracy: 0.4389 Epoch 5: val_accuracy improved from 0.44375 to 0.47661, saving model to mobilenetv3_1.h5 472/472 [==============================] - 184s 391ms/step - loss: 1.2213 - accuracy: 0.4389 - val_loss: 1.1653 - val_accuracy: 0.4766 - lr: 0.0100 Epoch 6/20 472/472 [==============================] - ETA: 0s - loss: 1.2211 - accuracy: 0.4378 Epoch 6: val_accuracy did not improve from 0.47661 472/472 [==============================] - 172s 364ms/step - loss: 1.2211 - accuracy: 0.4378 - val_loss: 1.2145 - val_accuracy: 0.4302 - lr: 0.0100 Epoch 7/20 472/472 [==============================] - ETA: 0s - loss: 1.2170 - accuracy: 0.4377 Epoch 7: val_accuracy did not improve from 0.47661 472/472 [==============================] - 167s 354ms/step - loss: 1.2170 - accuracy: 0.4377 - val_loss: 1.1858 - val_accuracy: 0.4581 - lr: 0.0100 Epoch 8/20 472/472 [==============================] - ETA: 0s - loss: 1.2119 - accuracy: 0.4498Restoring model weights from the end of the best epoch: 5. Epoch 8: val_accuracy did not improve from 0.47661 Epoch 8: ReduceLROnPlateau reducing learning rate to 0.0019999999552965165. 472/472 [==============================] - 186s 394ms/step - loss: 1.2119 - accuracy: 0.4498 - val_loss: 1.2090 - val_accuracy: 0.4480 - lr: 0.0100 Epoch 8: early stopping
# ModelCheckpoint is used to save the model at certain intervals
checkpoint_mobnetv3_2 = ModelCheckpoint("mobilenetv3_2.h5", # file path where the model will be saved
monitor='val_accuracy', # means the callback monitors the validation accuracy
verbose=1, # model is saved only when the metric (val_acc) has improved
save_best_only=True,
mode='max')
# Configured callbacks will be passed to the model during training
callbacks_list_mobnetv3_2 = [early_stopping, checkpoint_mobnetv3_2]#, reduce_learningrate]
# Number of iterations over the entire dataset that the training process should run
epochs = 20
# Fitting the model with required parameters above, saving model into history_resnet variable
history_mobilenetv3_2 = model.fit(
train_set_pre,
steps_per_epoch = train_set_pre.samples // train_set_pre.batch_size,
validation_data = validation_set_pre,
validation_steps = validation_set_pre.samples // validation_set_pre.batch_size,
epochs = epochs,
verbose = 1,
callbacks = callbacks_list_mobnetv3_2
)
Epoch 1/20 472/472 [==============================] - ETA: 0s - loss: 1.1997 - accuracy: 0.4504 Epoch 1: val_accuracy improved from -inf to 0.47500, saving model to mobilenetv3_2.h5 472/472 [==============================] - 167s 354ms/step - loss: 1.1997 - accuracy: 0.4504 - val_loss: 1.1579 - val_accuracy: 0.4750 Epoch 2/20 472/472 [==============================] - ETA: 0s - loss: 1.1946 - accuracy: 0.4586 Epoch 2: val_accuracy improved from 0.47500 to 0.47964, saving model to mobilenetv3_2.h5 472/472 [==============================] - 178s 378ms/step - loss: 1.1946 - accuracy: 0.4586 - val_loss: 1.1509 - val_accuracy: 0.4796 Epoch 3/20 472/472 [==============================] - ETA: 0s - loss: 1.1850 - accuracy: 0.4665 Epoch 3: val_accuracy did not improve from 0.47964 472/472 [==============================] - 207s 439ms/step - loss: 1.1850 - accuracy: 0.4665 - val_loss: 1.1576 - val_accuracy: 0.4778 Epoch 4/20 472/472 [==============================] - ETA: 0s - loss: 1.1855 - accuracy: 0.4688 Epoch 4: val_accuracy improved from 0.47964 to 0.49738, saving model to mobilenetv3_2.h5 472/472 [==============================] - 192s 408ms/step - loss: 1.1855 - accuracy: 0.4688 - val_loss: 1.1376 - val_accuracy: 0.4974 Epoch 5/20 472/472 [==============================] - ETA: 0s - loss: 1.1857 - accuracy: 0.4657 Epoch 5: val_accuracy did not improve from 0.49738 472/472 [==============================] - 168s 354ms/step - loss: 1.1857 - accuracy: 0.4657 - val_loss: 1.1407 - val_accuracy: 0.4891 Epoch 6/20 472/472 [==============================] - ETA: 0s - loss: 1.1822 - accuracy: 0.4709 Epoch 6: val_accuracy did not improve from 0.49738 472/472 [==============================] - 158s 334ms/step - loss: 1.1822 - accuracy: 0.4709 - val_loss: 1.1483 - val_accuracy: 0.4879 Epoch 7/20 472/472 [==============================] - ETA: 0s - loss: 1.1791 - accuracy: 0.4686Restoring model weights from the end of the best epoch: 4. Epoch 7: val_accuracy did not improve from 0.49738 472/472 [==============================] - 150s 317ms/step - loss: 1.1791 - accuracy: 0.4686 - val_loss: 1.1406 - val_accuracy: 0.4879 Epoch 7: early stopping
import tensorflow as tf
import tensorflow_hub as hub
# Ensure that the SEBlock class is defined exactly as it was when the model was saved
class SEBlock(tf.keras.layers.Layer):
def __init__(self, ratio=16):
super(SEBlock, self).__init__()
self.ratio = ratio
# Removing the GlobalAveragePooling2D layer since it's not needed for 2D inputs
def build(self, input_shape):
num_channels = input_shape[-1]
self.fc1 = tf.keras.layers.Dense(num_channels // self.ratio, activation='relu')
self.fc2 = tf.keras.layers.Dense(num_channels, activation='sigmoid')
def call(self, inputs):
# Using `inputs` instead of `x`
x = self.fc1(inputs)
x = self.fc2(x)
return x * inputs
# Load the model with custom object scope
with tf.keras.utils.custom_object_scope({'KerasLayer': hub.KerasLayer, 'SEBlock': SEBlock}):
mobilenetv3_2 = tf.keras.models.load_model("mobilenetv3_2.h5")
# Evaluate the model on the test data
test_loss, test_accuracy = mobilenetv3_2.evaluate(test_set_pre, verbose=1)
# Print the results
print("Test Loss:", test_loss)
print("Test Accuracy:", test_accuracy)
4/4 [==============================] - 2s 219ms/step - loss: 1.0917 - accuracy: 0.5078 Test Loss: 1.0916602611541748 Test Accuracy: 0.5078125
# Create an instance of ImageDataGenerator with additional image transformations (Horizontal Flip and rotation)
datagen = ImageDataGenerator(
rescale = 1. / 255,
horizontal_flip = True,
vertical_flip = False,
width_shift_range=0.2,
height_shift_range=0.2,
rotation_range = 25,
zoom_range=0.5,
brightness_range=(0.,2.),
shear_range=0.3,
fill_mode='nearest')
# Set the target size to 48x48 again
img_size = (48, 48)
# Creating dataloader for train for Grayscale dataset
train_set_gray = datagen.flow_from_directory(folder_path + "train",
target_size = (img_size),
color_mode = 'grayscale',
batch_size = batch_size,
class_mode = 'categorical',
shuffle = True)
# Creating dataloader for validation for Grayscale dataset
validation_set_gray = datagen.flow_from_directory(folder_path + "validation",
target_size = (img_size),
color_mode = 'grayscale',
batch_size = batch_size,
class_mode = 'categorical',
shuffle = True)
# Creating dataloader for test for Grayscale dataset
test_set_gray = datagen.flow_from_directory(folder_path + "test",
target_size = (img_size),
color_mode = 'grayscale',
batch_size = batch_size,
class_mode = 'categorical',
shuffle = True)
Found 15109 images belonging to 4 classes. Found 4977 images belonging to 4 classes. Found 128 images belonging to 4 classes.
# Clearing backend
backend.clear_session()
# Reset seed for number generators using reset_seed() function
reset_seed(fixed_seed)
Random number generators seed has been set to 42
# Load the saved model if notebook session gets disconnected
model2_gray_da = load_model("model2_grayscale.h5")
# ModelCheckpoint is used to save the model at certain intervals
checkpoint_best_da = ModelCheckpoint("best_da.h5", # file path where the model will be saved
monitor='val_accuracy', # means the callback monitors the validation accuracy
verbose=1, # model is saved only when the metric (val_acc) has improved
save_best_only=True,
mode='max')
# Stop the training process early if a monitored metric has stopped improving
early_stopping = EarlyStopping(monitor = 'val_loss', # monitors the validation loss
min_delta = 0, # any positive change is considered as an improvement
patience = 3, # training will be stopped if no improvement is seen for 3 epochs
verbose = 1,
restore_best_weights = True # weights are reverted to best value after early stopping
)
# Configured callbacks will be passed to the model during training
callbacks_list_mobnetv3 = [early_stopping, checkpoint_best_da]#, reduce_learningrate]
# Number of iterations over the entire dataset that the training process should run
epochs = 20
# Train the model
history = model2_gray_da.fit(
train_set_gray,
steps_per_epoch=train_set_gray.samples // train_set_gray.batch_size,
validation_data=validation_set_gray,
validation_steps=validation_set_gray.samples // validation_set_gray.batch_size,
epochs=epochs
)
Epoch 1/20 472/472 [==============================] - 100s 210ms/step - loss: 1.1809 - accuracy: 0.4630 - val_loss: 1.1547 - val_accuracy: 0.4879 Epoch 2/20 472/472 [==============================] - 108s 229ms/step - loss: 1.1257 - accuracy: 0.5037 - val_loss: 1.2045 - val_accuracy: 0.4333 Epoch 3/20 472/472 [==============================] - 132s 280ms/step - loss: 1.1144 - accuracy: 0.5008 - val_loss: 1.1813 - val_accuracy: 0.4460 Epoch 4/20 472/472 [==============================] - 102s 216ms/step - loss: 1.0910 - accuracy: 0.5213 - val_loss: 1.0842 - val_accuracy: 0.5032 Epoch 5/20 472/472 [==============================] - 142s 302ms/step - loss: 1.0754 - accuracy: 0.5246 - val_loss: 1.1057 - val_accuracy: 0.4990 Epoch 6/20 472/472 [==============================] - 139s 295ms/step - loss: 1.0696 - accuracy: 0.5228 - val_loss: 1.1310 - val_accuracy: 0.4821 Epoch 7/20 472/472 [==============================] - 113s 240ms/step - loss: 1.0529 - accuracy: 0.5360 - val_loss: 1.0399 - val_accuracy: 0.5427 Epoch 8/20 472/472 [==============================] - 116s 246ms/step - loss: 1.0553 - accuracy: 0.5380 - val_loss: 1.0099 - val_accuracy: 0.5579 Epoch 9/20 472/472 [==============================] - 97s 206ms/step - loss: 1.0425 - accuracy: 0.5439 - val_loss: 1.1024 - val_accuracy: 0.5050 Epoch 10/20 472/472 [==============================] - 90s 190ms/step - loss: 1.0370 - accuracy: 0.5496 - val_loss: 1.0711 - val_accuracy: 0.5125 Epoch 11/20 472/472 [==============================] - 96s 204ms/step - loss: 1.0268 - accuracy: 0.5565 - val_loss: 1.0818 - val_accuracy: 0.5095 Epoch 12/20 472/472 [==============================] - 101s 214ms/step - loss: 1.0181 - accuracy: 0.5556 - val_loss: 1.0240 - val_accuracy: 0.5498 Epoch 13/20 472/472 [==============================] - 109s 231ms/step - loss: 1.0078 - accuracy: 0.5617 - val_loss: 1.0130 - val_accuracy: 0.5466 Epoch 14/20 472/472 [==============================] - 98s 207ms/step - loss: 1.0102 - accuracy: 0.5618 - val_loss: 1.0570 - val_accuracy: 0.5373 Epoch 15/20 472/472 [==============================] - 102s 217ms/step - loss: 1.0057 - accuracy: 0.5578 - val_loss: 1.0006 - val_accuracy: 0.5603 Epoch 16/20 472/472 [==============================] - 108s 229ms/step - loss: 1.0074 - accuracy: 0.5594 - val_loss: 1.0061 - val_accuracy: 0.5599 Epoch 17/20 472/472 [==============================] - 104s 220ms/step - loss: 1.0051 - accuracy: 0.5643 - val_loss: 0.9993 - val_accuracy: 0.5756 Epoch 18/20 472/472 [==============================] - 123s 261ms/step - loss: 0.9910 - accuracy: 0.5707 - val_loss: 1.0433 - val_accuracy: 0.5317 Epoch 19/20 472/472 [==============================] - 120s 253ms/step - loss: 0.9944 - accuracy: 0.5705 - val_loss: 0.9757 - val_accuracy: 0.5802 Epoch 20/20 472/472 [==============================] - 108s 228ms/step - loss: 0.9810 - accuracy: 0.5748 - val_loss: 1.0382 - val_accuracy: 0.5367
# Evaluate the model on the test data using Grayscale model
test_loss, test_accuracy = model2_gray_da.evaluate(test_set_gray, verbose=1)
# Print the results
print("Test Loss:", test_loss)
print("Test Accuracy:", test_accuracy)
4/4 [==============================] - 0s 80ms/step - loss: 1.0238 - accuracy: 0.5703 Test Loss: 1.0238081216812134 Test Accuracy: 0.5703125
# Plotting the Training and Validation accuracies of CNN model 2 and Grayscale images
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.title('Best CNN Model with Data Augmentation')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc = 'upper left')
# Display the plot
plt.show()
Observations and Insights:
- "surprise" had the highest precision (0.83) and a good recall rate (0.75).
- "happy" had the second best performance with a precision of 0.59 and a recall of 0.72.
- "neutral" class stands in third, with a precision of 0.30 and a recall of 0.28
- "sad" has performed the worst, with 0.23 precision, 0.22 recall